Feature #17258: Oneshot Branch Coverage

• Status: Open
• Assignee: -
• Target version: -

[jemmai (Jemma Issroff)]

Description

I'd like to propose adding a new option to Coverage.start: oneshot branch coverage. It will be analogous to the existing "oneshot lines coverage". Oneshot branch coverage will record the first execution of each branch.

Background

Two years ago, oneshot lines coverage was added to the Coverage module. It records the first execution of every line, and returns the line numbers of newly executed lines.

As the original feature request described, this reduced overhead to calculations for line coverage. It solved two primary use cases, more easily enabling both coverage measurement in production and coverage measurement in CPU-intensive programs.

Line coverage alone does not actually indicate full coverage of all branch options. See this blog post for more description about why not. It would therefore make sense that for the same reasons as oneshot line coverage was implemented, we should implement oneshot branch coverage to tell the full story of coverage in a way that enables coverage measurement in production and CPU-intensive programs.

Proposal

Coverage.start(oneshot_branches: true)

We add the ability to pass oneshot_branches: true to Coverage.start to return a Hash that would indicate if branches had been executed or not. It would use the same [BRANCH_TYPE, UNIQUE_ID, START_LINE_NUMBER, START_COLUMN_NUMBER, END_LINE_NUMBER, END_COLUMN_NUMBER] format to uniquely identify branches as the branches: true keyword argument already returns.

Here is an example:

test.rb

require "coverage"
Coverage.start(oneshot_branches: true)
load "target.rb"
puts Coverage.result

target.rb

2.times do
  if 1 == 0
    puts :match
  else
    puts :not_match
  end
end

Running test.rb would output the following:

$ ruby test.rb
not_match
not_match
{
  "target.rb"=>{
    :oneshot_branches=>{
      [:if, 0, 2, 2, 6, 5]=>{
        [:then, 1, 3, 4, 3, 15]=>0,
        [:else, 2, 5, 4, 5, 19]=>1
      }
    }
  }
}

Discussion

If we've deemed oneshot line coverage necessary, and we know that line coverage does not accurately demonstrate full coverage, it only makes sense to also implement oneshot branch coverage.

Reiterating the points made in the feature request for oneshot line coverage, there are two main drivers for oneshot branch coverage:

Branch coverage measurement in production

Oneshot branch coverage will allow us to print one shot branch coverage to logs and indicate where there is dead code for branch conditions in which one (or more) conditions never actually occur.

Branch coverage in CPU-intensive programs

There is an overhead to running coverage measurements on test suites. It is encouraged to run branch coverage as well as line coverage, for the reasons indicated above. Oneshot branch coverage will allow us to run branch coverage easily in CPU-intensive programs.

Summary

Analogous to oneshot_lines, exposing a oneshot_branches option for Coverage.start will allow for a fuller picture of coverage in a less CPU-intensive way.