https://redmine.ruby-lang.org/https://redmine.ruby-lang.org/favicon.ico?17113305112013-04-04T21:50:34ZRuby Issue Tracking SystemRuby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=382152013-04-04T21:50:34ZEregon (Benoit Daloze)
<ul><li><strong>Category</strong> set to <i>core</i></li><li><strong>Status</strong> changed from <i>Open</i> to <i>Closed</i></li></ul><p>Thread#[] and friends access <em>Fiber</em>-local variables, as the doc says:</p>
<p>(Attribute Reference---Returns the value of a fiber-local variable (current<br>
thread's root fiber if not explicitely inside a Fiber), using either a symbol<br>
or a string name.)</p>
<p>This is unfortunate, as indeed one might expect it to be <em>thread</em>-local,<br>
but it has been made fiber-local for safety.</p>
<p>In ruby >= 2.0.0, you have thread_variable_{get,set} and such for manipulating <em>thread</em>-local variables.<br>
Not as nice, but at least the API is there. See <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: Thread locals don't work inside Enumerator (Closed)" href="https://redmine.ruby-lang.org/issues/7097">#7097</a>.</p> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=382592013-04-05T18:27:50Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/38259/diff?detail_id=27620">diff</a>)</li><li><strong>Status</strong> changed from <i>Closed</i> to <i>Open</i></li></ul><p>According to that rubinius issue tracker, it seems a feature request for <code>Fiber#[]</code> and <code>Fiber#[]=</code>.</p> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=382612013-04-05T20:06:12ZEregon (Benoit Daloze)
<ul></ul><p>Ah, sorry, I thought it was only misunderstanding of Thread#[] and such.</p> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=383282013-04-07T19:13:59Zhalorgium (Tim Carey-Smith)ruby@spork.in
<ul></ul><p>Yup, I am sorry if that was not clear!</p>
<p>I also am very interested in being able to get the backtrace of a <code>Fiber</code> too.</p>
<p>Should I open a new issue for <code>Fiber#backtrace</code>?</p> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=383852013-04-09T16:33:43Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul></ul><p>One ticket, one issue, please.</p> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=386092013-04-16T17:03:40Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul><li><strong>File</strong> <a href="/attachments/3664">0001-cont.c-fiber-local-accessors.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/3664/0001-cont.c-fiber-local-accessors.patch">0001-cont.c-fiber-local-accessors.patch</a> added</li></ul><p>More tests may be needed.</p> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=387392013-04-19T15:50:02Zhalorgium (Tim Carey-Smith)ruby@spork.in
<ul></ul><p>I realised that this might be better in common-ruby.<br>
I originally proposed the idea with RBX.<br>
Could someone move it?</p> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=387412013-04-19T16:23:14Zduerst (Martin Dürst)duerst@it.aoyama.ac.jp
<ul></ul><p>On 2013/04/19 15:50, halorgium (Tim Carey-Smith) wrote:</p>
<blockquote>
<p>Issue <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Support accessing Fiber-locals and backtraces for a Fiber (Closed)" href="https://redmine.ruby-lang.org/issues/8215">#8215</a> has been updated by halorgium (Tim Carey-Smith).</p>
<p>I realised that this might be better in common-ruby.</p>
</blockquote>
<p>This isn't issue specific: I propose that just for the moment, issues<br>
stay where they are. Once the overall directions are sorted out, we can<br>
organize a general campaign to move issues wherever necessary. If we can<br>
avoid it, we don't want to pollute each and every issue with individual<br>
move requests.</p>
<p>Regards, Martin.</p> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=396292013-06-02T15:45:22Zzzak (zzak _)
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Assigned</i></li><li><strong>Assignee</strong> set to <i>nobu (Nobuyoshi Nakada)</i></li></ul> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=836562020-01-05T22:51:37Zioquatix (Samuel Williams)samuel@oriontransfer.net
<ul></ul><p><a class="user active user-mention" href="https://redmine.ruby-lang.org/users/13">@matz (Yukihiro Matsumoto)</a> I agree with adding all three APIs, <code>Fiber#[]</code>, <code>Fiber#[]=</code> and <code>Fiber#backtrace</code>. Can you let me know if you are happy with these additions?</p> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=836572020-01-05T22:51:51Zioquatix (Samuel Williams)samuel@oriontransfer.net
<ul><li><strong>Assignee</strong> changed from <i>nobu (Nobuyoshi Nakada)</i> to <i>ioquatix (Samuel Williams)</i></li></ul> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=836782020-01-06T18:22:11ZEregon (Benoit Daloze)
<ul></ul><p><code>Fiber#[]</code> and <code>Fiber#[]=</code> sounds fine, but what if somebody does:</p>
<p><code>some_fiber = Fiber.new { ... }; Thread.new { some_fiber[:fiber_local] }</code>?</p>
<p>I think that should raise or not be possible.<br>
If it would return the value, it would imply synchronization on every access to fiber locals which seems unfortunate.</p>
<p>By making the API <code>Fiber.[]</code>, we can avoid that entirely and have true fiber-locals, which can only be accessed by that Fiber:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">Fiber</span><span class="p">[</span><span class="ss">:my_fiber_local</span><span class="p">]</span> <span class="o">=</span> <span class="mi">42</span>
<span class="n">value</span> <span class="o">=</span> <span class="no">Fiber</span><span class="p">[</span><span class="ss">:my_fiber_local</span><span class="p">]</span>
<span class="c1"># no way to access fiber locals of another Fiber</span>
</code></pre>
<p>I think for new APIs we should take the chance to make them only possible to use correctly.<br>
We could even finally have Fiber and Thread local in a consistent way:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># access Fiber-local</span>
<span class="no">Fiber</span><span class="p">[</span><span class="ss">:my_fiber_local</span><span class="p">]</span>
<span class="c1"># access Thread-local</span>
<span class="no">Thread</span><span class="p">[</span><span class="ss">:my_thread_local</span><span class="p">]</span>
</code></pre> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=836792020-01-06T20:50:57Zioquatix (Samuel Williams)samuel@oriontransfer.net
<ul></ul><p><a class="user active user-mention" href="https://redmine.ruby-lang.org/users/772">@Eregon (Benoit Daloze)</a> I agree with your points. I respect that you've studied a lot in your thesis so ultimately I'll defer to your judgement. But let me explain a bit more.</p>
<p>The reason for expanding the <code>Fiber#</code> interface is so that tools like <code>async</code> can show better debugging of all fibers.</p>
<p>Ideally it can show the backtrace, and allow the user to check fiber locals (and maybe even get a list of keys for debugging purposes).</p>
<p>I'd also be okay with adding <code>Fiber.[]</code> and so on, but that's kind of a separate issue.</p>
<p>Thread safety is not my concern, the function can be marked as thread unsafe, and maybe we can add detection of this and warn against it.</p> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=851882020-04-19T05:21:45Zioquatix (Samuel Williams)samuel@oriontransfer.net
<ul></ul><p><a class="user active user-mention" href="https://redmine.ruby-lang.org/users/13">@matz (Yukihiro Matsumoto)</a> do you mind checking this when you have time? Especially <code>Fiber#backtrace</code> is super important for debugging issues with fibers.</p> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=852162020-04-21T02:18:39Zioquatix (Samuel Williams)samuel@oriontransfer.net
<ul></ul><p>Here is a quick hack I used to add <code>Fiber.backtrace</code> to aid in debugging. I'd say, don't use it in production, but I did :p</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">module</span> <span class="nn">Fiber::Backtrace</span>
<span class="k">def</span> <span class="nf">yield</span>
<span class="no">Fiber</span><span class="p">.</span><span class="nf">current</span><span class="p">.</span><span class="nf">backtrace</span> <span class="o">=</span> <span class="nb">caller</span>
<span class="k">super</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">class</span> <span class="nc">Fiber</span>
<span class="nb">attr_accessor</span> <span class="ss">:backtrace</span>
<span class="k">class</span> <span class="o"><<</span> <span class="nb">self</span>
<span class="n">prepend</span> <span class="no">Fiber</span><span class="o">::</span><span class="no">Backtrace</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=852772020-04-24T08:37:55ZEregon (Benoit Daloze)
<ul></ul><p><a class="user active user-mention" href="https://redmine.ruby-lang.org/users/3344">@ioquatix (Samuel Williams)</a> Please create a separate feature ticket for <code>Fiber#backtrace</code>, as already asked by nobu in <a href="https://bugs.ruby-lang.org/issues/8215#note-5" class="external">https://bugs.ruby-lang.org/issues/8215#note-5</a><br>
I'd like to keep the discussion here about Fiber locals, not mix it with an unrelated method.<br>
A separate ticket is better so e.g. the discussion is clear and focused, that ticket can be closed when implemented, the reference from NEWS will be much less confusing, etc.</p>
<p>Back to the fiber locals discussion:</p>
<blockquote>
<p>Thread safety is not my concern, the function can be marked as thread unsafe, and maybe we can add detection of this and warn against it.</p>
</blockquote>
<p>It's of course unacceptable if a Ruby implementation behaves unsafely (e.g., crashes or loses fiber local variable) for this, so it is a relevant concern for this issue.</p>
<p>Another concern is this needs <code>Fiber.current</code>, which is only available after <code>require "fiber"</code>.<br>
I think that could be confusing, so I'd suggest making <code>Fiber.current</code> always available if we go that way.</p>
<p>I think <code>Fiber[:foo]</code> and <code>Fiber[:foo] = ...</code> is a better API because it avoids this problem entirely, and gives the possibility to have clean Thread and Fiber locals APIs.<br>
We could also add a check in <code>Fiber#[]</code> but that would read the current Fiber a second time, and move it to a runtime error instead of simply not being possible with <code>Fiber[:foo]</code>.<br>
Maybe the check is not so important since I guess <code>Thread#[]</code> will still need to work for a long while (but maybe we could deprecate cross-thread/cross-fiber access), but for new APIs I think a good safe design is worth considering.</p>
<p>If you think we need the ability to access another Fiber's locals, could you show some examples?</p> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=852802020-04-24T09:05:14ZEregon (Benoit Daloze)
<ul></ul><p>To make it clearer about "clean API": if we have Fiber#[] and Thread#[] then Thread#[] is just deceptive and doesn't do anything different.<br>
OTOH, with Fiber[:foo] and Thread[:foo] we can actually have the intuitive semantics.</p> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=852832020-04-24T10:23:00Zioquatix (Samuel Williams)samuel@oriontransfer.net
<ul></ul><blockquote>
<p>It's of course unacceptable if a Ruby implementation behaves unsafely (e.g., crashes or loses fiber local variable) for this, so it is a relevant concern for this issue.</p>
</blockquote>
<p>We already have situations in JRuby and TruffleRuby where unsynchronized access to shared mutable state can cause a crash or data corruption. So why is this different?</p>
<p>If you believe it's important, then I defer to your judgement. However, from my point of view:</p>
<ul>
<li>This method should not be invoked across threads.</li>
<li>This method should not need to be thread safe.</li>
<li>The GVL might make it safe but it shouldn't be guaranteed.</li>
<li>Maybe we need annotations or other documentation to explain this.</li>
</ul>
<blockquote>
<p>If you think we need the ability to access another Fiber's locals, could you show some examples?</p>
</blockquote>
<p>The reason to access other fiber locals is mostly for debugging purposes. In Async, we make a list of tasks:</p>
<pre><code> 4874m43s info: Falcon::Controller::Proxy [oid=0x2b28b0f0d558] [pid=324886] [2020-04-24 09:55:55 +0000]
| #<Async::Reactor:0x2b28b10316c8 1 children running>
| #<Async::Task:0x2b28b1031498 (running)>
| #<Async::Task:0x2b28b10312e0 waiting for signal USR1 (running)>
| #<Async::Task:0x2b28b1030930 accepting secure connection #<Addrinfo: [::ffff:34.228.115.75]:26564 TCP> (running)>
| #<Async::Task:0x2b28b1079b1c connected to #<Addrinfo: /srv/http/www.oriontransfer.co.nz/application.ipc SOCK_STREAM> [fd=17] (complete)>
| #<Async::Task:0x2b28b109a7b8 h2 reading data for Async::HTTP::Protocol::HTTP2::Client. (running)>
| #<Async::Task:0x2b28b10869e8 Reading h2 requests for Async::HTTP::Protocol::HTTP2::Server. (complete)>
| #<Async::Task:0x2b28b13a2dc0 connected to #<Addrinfo: /srv/http/www.codeotaku.com/application.ipc SOCK_STREAM> [fd=16] (complete)>
| #<Async::Task:0x2b28b12fb8f4 h2 reading data for Async::HTTP::Protocol::HTTP2::Client. (running)>
| #<Async::Task:0x2b28b0fefe30 Reading h2 requests for Async::HTTP::Protocol::HTTP2::Server. (running)>
| #<Async::Task:0x2b28b1004b64 h2 reading data for Async::HTTP::Protocol::HTTP2::Server. (running)>
</code></pre>
<p>When something is going wrong, the ability to dig into the fiber state is very useful. My plan is to make debugging tools to expose this more easily but I can't do it if there are no methods to access the state.</p> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=852852020-04-24T12:08:18ZEregon (Benoit Daloze)
<ul></ul><blockquote>
<p>We already have situations in JRuby and TruffleRuby where unsynchronized access to shared mutable state can cause a crash or data corruption. So why is this different?</p>
</blockquote>
<p>Let's not introduce more bugs (and IMHO those bugs should be fixed), especially if these methods are supposed to access <em>local</em> state, which means state you may not access from any other Thread.</p>
<p>It's probably fine to access locals of another Fiber of the same Thread though, that's not racy since Fibers of a Thread don't run simultaneously.<br>
So I'd be OK with that, if we raise when trying to access locals of a Fiber belonging to a different Thread.</p>
<p>Is there a reason you want to store this state in Fiber locals and not e.g., in instance variables of the Fiber object?</p> Ruby master - Feature #8215: Support accessing Fiber-locals and backtraces for a Fiberhttps://redmine.ruby-lang.org/issues/8215?journal_id=869812020-08-08T23:11:25Zioquatix (Samuel Williams)samuel@oriontransfer.net
<ul><li><strong>Status</strong> changed from <i>Assigned</i> to <i>Closed</i></li></ul><p>It looks like we can implement <code>Fiber#backtrace</code>:</p>
<p><a href="https://bugs.ruby-lang.org/issues/16815" class="external">https://bugs.ruby-lang.org/issues/16815</a></p>
<p>and in addition we should continue the discussion about Fiber locals:</p>
<p><a href="https://bugs.ruby-lang.org/issues/13893" class="external">https://bugs.ruby-lang.org/issues/13893</a></p>