https://redmine.ruby-lang.org/https://redmine.ruby-lang.org/favicon.ico?17113305112020-04-10T17:20:23ZRuby Issue Tracking SystemRuby master - Bug #16776: Regression in coverage libraryhttps://redmine.ruby-lang.org/issues/16776?journal_id=850332020-04-10T17:20:23Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul></ul><p>Thank you for your report. The reason why the method execution is missed is because:</p>
<ul>
<li>coverage measurement is based on TracePoint mechanism (since 2.6), and</li>
<li>no TracePoint event is fired during the execution of TracePoint hooks (to avoid infinite chain of hooks).</li>
</ul>
<p>So, this is a limitation of TracePoint. Currently, the combination of multiple TracePoint hooks works not so great; they are completely exclusive.</p>
<p>This issue may be solved by improving TracePoint mechanism, but it is expensive to implement and <a class="user active user-mention" href="https://redmine.ruby-lang.org/users/17">@ko1 (Koichi Sasada)</a> is negative. How severe is this issue for you?</p> Ruby master - Bug #16776: Regression in coverage libraryhttps://redmine.ruby-lang.org/issues/16776?journal_id=850342020-04-10T17:25:09Zdeivid (David RodrÃguez)
<ul></ul><p>I see.</p>
<p>It's a severe as "I can no longer track code coverage on my library that makes heavy use of TracePoint" is considered.</p>
<p>(Edited the message to add thank you for your response and your work in general :))</p> Ruby master - Bug #16776: Regression in coverage libraryhttps://redmine.ruby-lang.org/issues/16776?journal_id=850692020-04-12T05:32:31Zmarcandre (Marc-Andre Lafortune)marcandre-ruby-core@marc-andre.ca
<ul></ul><p>I never realized this limitation. I imagine that for the vast majority of users it's a non issue, but for <code>byebug</code> it is quite serious. Would you consider using <code>deep-cover</code> instead?</p> Ruby master - Bug #16776: Regression in coverage libraryhttps://redmine.ruby-lang.org/issues/16776?journal_id=850712020-04-12T06:27:27Zmarcandre (Marc-Andre Lafortune)marcandre-ruby-core@marc-andre.ca
<ul></ul><p>I'm not super familiar with <code>TracePoint</code>, but at first sight <code>DeepCover</code> shouldn't generate any extra callbacks as the instrumenting is minimal, typically adding <code>$tracker[42] += 1</code> and taking care to never change the line numbers.</p>
<p>So it should work even with <code>TracePoint</code> enabled and shouldn't impact your tests. I tested quickly with DeepCover's clone mode and confirm that your test suite appear to pass :-)</p> Ruby master - Bug #16776: Regression in coverage libraryhttps://redmine.ruby-lang.org/issues/16776?journal_id=850832020-04-12T13:14:15ZEregon (Benoit Daloze)
<ul></ul><p>I'm surprised too the coverage stdlib gets hit by this issue.</p>
<p>Maybe coverage could use a second type of internal tracepoint to fix this issue (internal coverage handlers are likely pure C and so don't emit any extra event)?</p> Ruby master - Bug #16776: Regression in coverage libraryhttps://redmine.ruby-lang.org/issues/16776?journal_id=850842020-04-12T14:01:13Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul></ul><p>IMO, TracePoint hooks should have a priority. A TracePoint hook that is first enabled should monitor a hook code that is second enabled:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="mi">1</span><span class="p">:</span> <span class="no">TracePoint</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">:line</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">tp</span><span class="o">|</span>
<span class="mi">2</span><span class="p">:</span> <span class="nb">p</span> <span class="p">[</span><span class="ss">:parent</span><span class="p">,</span> <span class="n">tp</span><span class="p">]</span>
<span class="mi">3</span><span class="p">:</span> <span class="k">end</span><span class="p">.</span><span class="nf">enable</span> <span class="k">do</span>
<span class="mi">4</span><span class="p">:</span> <span class="no">TracePoint</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">:line</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">tp</span><span class="o">|</span>
<span class="mi">5</span><span class="p">:</span> <span class="nb">p</span> <span class="p">[</span><span class="ss">:child</span><span class="p">,</span> <span class="n">tp</span><span class="p">]</span>
<span class="mi">6</span><span class="p">:</span> <span class="k">end</span><span class="p">.</span><span class="nf">enable</span> <span class="k">do</span>
<span class="mi">7</span><span class="p">:</span> <span class="nb">p</span> <span class="ss">:main</span>
<span class="mi">8</span><span class="p">:</span> <span class="k">end</span>
<span class="mi">9</span><span class="p">:</span> <span class="k">end</span>
</code></pre>
<pre><code>$ ruby t.rb
[:parent, #<TracePoint:line@t.rb:4>]
[:parent, #<TracePoint:line@<internal:trace_point>:96 in `new'>]
[:parent, #<TracePoint:line@<internal:trace_point>:196 in `enable'>]
[:child, #<TracePoint:line@t.rb:7>]
[:parent, #<TracePoint:line@t.rb:7>]
:main
</code></pre>
<p>I expect <code>[:parent, #<TracePoint:line@t.rb:5>]</code> before <code>[:child, #<TracePoint:line@t.rb:7>]</code>. <a class="user active user-mention" href="https://redmine.ruby-lang.org/users/17">@ko1 (Koichi Sasada)</a>, what do you think?</p> Ruby master - Bug #16776: Regression in coverage libraryhttps://redmine.ruby-lang.org/issues/16776?journal_id=850852020-04-12T14:04:42Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul></ul><p>Eregon (Benoit Daloze) wrote in <a href="#note-5">#note-5</a>:</p>
<blockquote>
<p>Maybe coverage could use a second type of internal tracepoint to fix this issue (internal coverage handlers are likely pure C and so don't emit any extra event)?</p>
</blockquote>
<p>Yes, it is the plan B that I have. I don't like it as it is very ad-hoc, though. If <a class="user active user-mention" href="https://redmine.ruby-lang.org/users/17">@ko1 (Koichi Sasada)</a> rejects the priority, I'll give it a try to implement.</p> Ruby master - Bug #16776: Regression in coverage libraryhttps://redmine.ruby-lang.org/issues/16776?journal_id=851772020-04-18T14:31:12Zdeivid (David RodrÃguez)
<ul></ul><p>marcandre (Marc-Andre Lafortune) wrote in <a href="#note-3">#note-3</a>:</p>
<blockquote>
<p>I never realized this limitation. I imagine that for the vast majority of users it's a non issue, but for <code>byebug</code> it is quite serious. Would you consider using <code>deep-cover</code> instead?</p>
</blockquote>
<p>It sounds like the plan is to fix this one way or another, so I'll wait for now. But thanks for pointing me to deep-cover, it sounds interesting!</p> Ruby master - Bug #16776: Regression in coverage libraryhttps://redmine.ruby-lang.org/issues/16776?journal_id=948622021-11-24T05:33:54Zhsbt (Hiroshi SHIBATA)hsbt@ruby-lang.org
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Assigned</i></li><li><strong>Assignee</strong> set to <i>mame (Yusuke Endoh)</i></li></ul> Ruby master - Bug #16776: Regression in coverage libraryhttps://redmine.ruby-lang.org/issues/16776?journal_id=948652021-11-24T07:26:58Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul><li><strong>Assignee</strong> changed from <i>mame (Yusuke Endoh)</i> to <i>ko1 (Koichi Sasada)</i></li></ul> Ruby master - Bug #16776: Regression in coverage libraryhttps://redmine.ruby-lang.org/issues/16776?journal_id=951562021-12-06T02:53:37Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-5 priority-4 priority-default closed" href="/issues/15912">Feature #15912</a>: Allow some reentrancy during TracePoint events</i> added</li></ul>