Feature #10634: Baselining with Benchmark

• Status: Open
• Assignee: -
• Target version: -

[rklemme (Robert Klemme)]

Add a method Benchmark::Report#baseline. This does not report measurements but stores them as a baseline value in the Benchmark::Report instance. This baseline is nil initially. Any invocation of Benchmark::Report#report will do one of two things:

1. If baseline is nil, report as today.
2. If baseline is set, report the difference of the current measurement (Benchmark::Tms) - the baseline value.

We use this to subtract effort for a baseline to get at the net effort. This is how code might look like:

Benchmark.bm do |x|
  x.baseline do
    REP.times {}
  end

  x.report "m" do
    REP.times { f() }
  end
end

Output could be something like this:

       user     system      total        real
m  0.220000   0.000000   0.220000 (  0.221528)

Note the absence of any output for the baseline invocation.

Notes:

1. Return values of #report, #bm and #bmbm are not changed, i.e. they still contain the raw measurement (i.e. without subtracting the baseline).
2. We probably want to add another method #clear_baseline to set the baseline value back to nil.
3. A reasonable variant would be to add a second line to the output presenting measurement minus baseline like this:

       user     system      total        real
m  0.220000   0.000000   0.220000 (  0.221528) measured
m  0.120000   0.000000   0.120000 (  0.121497) normalized

[hsbt (Hiroshi SHIBATA)]

• Project changed from 14 to Ruby