Bug #12480: Restarting Coverage does not capture additional coverage for already loaded files

• Status: Closed
• Priority: Normal
• Assignee: mame (Yusuke Endoh)
• Target version: -
• ruby -v: 2.3.1
• Backport: 2.1: UNKNOWN, 2.2: UNKNOWN, 2.3: UNKNOWN

[grosser (Michael Grosser)]

I'm trying to combine coverage from before fork and after fork to make coverage reporting work in a forking test runner.
The problem I ran into is 2-fold:

• A: when forking, previous coverage is lost
• B: when restarting coverage, old files do not get added to

I could work around issue A by storing the old result and then merging it with the new result post fork, but issue B makes that impossible.

Please fix either A or B ...

Reproduction steps for A:

# reproduce.rb
require 'coverage'

Coverage.start
require_relative 'test'
a

fork do
  b
  new = Coverage.result
  puts "NEW: #{new}"
end


# test.rb
def a
  1
end

def b
  1
end

NEW: {"/Users/mgrosser/Code/tools/forking_test_runner/test.rb"=>[0, 0, nil, nil, 0, 1, nil]}
-> missing coverage information for method a

Reproduction steps for B:

# reproduce.rb
require 'coverage'

Coverage.start
require_relative 'test'
a
old = Coverage.result
Coverage.start
b
new = Coverage.result
puts "OLD: #{old} -- NEW: #{new}"

# test.rb
def a
  1
end

def b
  1
end

OLD: {"test.rb"=>[1, 1, nil, nil, 1, 0, nil]} -- NEW: {"test.rb"=>[]}
-> missing coverage information for method b

[mame (Yusuke Endoh)]

• Assignee set to mame (Yusuke Endoh)
• Status changed from Open to Feedback

Michael Grosser wrote:

• A: when forking, previous coverage is lost

This is intended. Consider the following program:

# reproduce.rb
require "coverage"

Coverage.start
require_relative "test"

def save_coverage
  r = Coverage.result

  # merge coverage data
  if File.readable?("coverage.dat")
    r2 = Marshal.load(File.binread("coverage.dat"))
    r = merge(r, r2)
  end

  File.binwrite("coverage.dat", Marshal.dump(r))
end

simple_test

fork do
  heavy_test_1
  save_coverage
end

heavy_test_2
save_coverage

Note that the simple_test method is executed once. If fork does not clear the previous coverage, the method will be wrongly counted twice.

• B: when restarting coverage, old files do not get added to

This is also intended. See #12220.

I recommend you not to use fork. If you want to do so at any cost, you can pass the coverage data via pipe and do manual merge:

require "coverage"
Coverage.start

require_relative "test"

a

r, w = IO.pipe("BINARY")
fork do
  w.close
  c1 = Marshal.load(r.read)

  b

  c2 = Coverage.result
  puts "NEW: #{ merge(c1, c2) }"
end

r.close
w << Marshal.dump(Coverage.result)
w.close

[grosser (Michael Grosser)]

Solved this by using Coverage.peek_result in the before-fork part of my code to capture the current state and then merged it after the fork is done :D

Feel free to close this now!

[mame (Yusuke Endoh)]

• Status changed from Feedback to Closed

Closing due to OP's request.

FYI: Since 2.6, coverage.so has supported Coverage.result(stop: false, clear: false). You may want to check it out.