Feature #19056
closedIntroduce `Fiber.annotation` for attaching messages to fibers.
Description
It's useful to know what a fiber is doing especially when they have a temporal execution (i.e. sockets connecting vs connected, binding vs accepting, queue popping, etc)
Let's introduce Fiber.annotate
and Fiber#annotation
for logging a short message attached to Fibers.
Fiber.annotate "Counting to 10"
10.times{|I| puts I}
# Fiber.current.annotation => "Counting to 10"
Pull Request: https://github.com/ruby/ruby/pull/6554
Files
Updated by ioquatix (Samuel Williams) about 2 years ago
- Tracker changed from Bug to Feature
- Backport deleted (
2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN)
Updated by ioquatix (Samuel Williams) about 2 years ago
- Description updated (diff)
Updated by ioquatix (Samuel Williams) about 2 years ago
A counter-proposal to introduce Fiber#name
, but this is different concept.
fiber = Fiber.current
fiber.name = "Processor"
while item = queue.pop
fiber.annotation = "Processing item #{item}..."
# ... work ...
end
In this case, Fiber#annotation
is related to the current activity of the Fiber, not its general name in relation to other fibers. However, I'm not against introducing Fiber#name
in another proposal.
This feature is already part of Async::Task
and is very useful for debugging programs with a lot of fibers, since we can visualise the activity of all the different fibers easily. I'll share a screenshot and/or video.
Updated by ioquatix (Samuel Williams) about 2 years ago
Here is a picture of an interactive debugger (async-debug) which uses similar annotations to present useful information about the state of the system.
@Eregon (Benoit Daloze) proposed that there should be a way to switch it on and off. I don't have a strong opinion about it- I've not run into any performance issues with Async::Task#annotation
.
That being said, we could introduce something like Fiber.annotate{..}
which is only evaluated if annotations are switched on (e.g. Fiber.annotations = true/false
).
Updated by matz (Yukihiro Matsumoto) about 2 years ago
For the record, I write down the concerns:
- I don't see the convenience of
Fiber.annotate
. Convince me. - Users need to understand how to use
Fiber#annotation
wisely. Need documents. - The state should be stacked, not overridden. How do you think?
- I have a bit of performance concern. Is it OK?
Matz.
Updated by mame (Yusuke Endoh) about 2 years ago
I think annotation per Fiber will cause conflict in usage between libraries. For example, consider the following library code.
def fetch_info_from_url(uri)
Fiber.new do
Fiber.annotate "fetching info from #{ uri }"
URI.open(uri) do |f|
f.each_line do |line|
Fiber.yield line
end
end
end
end
f = fetch_info_from_uri("example.com")
p f.resume #=> first line of example.com
f.annotation #=> "fetching info from example.com"
It should work perfect. And then, suppose uri gem has been updated to use Fiber.annotate
in URI.open(uri)
.
def URI.open(...)
...
Fiber.annotate "calling the user block"
yield io
...
end
Now f.annotation
will return "calling the user block"
instead of "fetching info from example.com"
.
What is the bad thing here? Must uri gem not use Fiber.annotate
? If so, what is allowed to use it?
Also, the message "calling the user block"
is correct in the context of uri gem, but not very informative for the application authors. However, it is difficult for uri gem to write a more appropriate message because it does not know the context of the entire application.
After all, only the application authors can use Fiber.annotate
, not library authors. If so, there seems little need to introduce it in the core.
Updated by ioquatix (Samuel Williams) about 2 years ago
I want to add annotations to default gems like Net::HTTP
so it would need to be in core, or at least some kind of hook. If it's too much to have a specific implementation, we could just make a general annotation method and then hook it up with a callback to the fiber scheduler or something.
Another use case: https://github.com/socketry/async/issues/78
Updated by ioquatix (Samuel Williams) over 1 year ago
I came back to this issue.
I don't see the convenience of
Fiber.annotate
. Convince me.
- Logging what the current fiber is doing.
- Recording what the current fiber is doing when an error or exception occurs.
- Real time debugger output (e.g. as shown in the screenshot).
Users need to understand how to use Fiber#annotation wisely. Need documents.
Yes, I am happy to include documentation about it.
The state should be stacked, not overridden. How do you think?
What about:
Fiber.annotate "x" do
Fiber.annotation # "x"
Fiber.annotate "y" do
Fiber.annotation # "y"
end
Fiber.annotation # "x"
end
I have a bit of performance concern. Is it OK?
It's a single pointer assignment so it should be minimal. Compared to "blocking operations" it should be irrelevant.
In order to try this out more generally, I created a gem. You can see the proposed implementation here: https://github.com/ioquatix/fiber-annotate/blob/main/lib/fiber/annotate.rb
The reason to make this a core interface, is so that default gems can use it.
Updated by Eregon (Benoit Daloze) over 1 year ago
Why not use a fiber-local variable for this? Then there is much less risk of conflict between libraries, and of course you can define convenience methods as you like.
Or as you implemented it in that gem, as an attribute + annotate(message) { ... }
.
IMO this is good to be in a gem. I don't think it needs to be in core.
Updated by ioquatix (Samuel Williams) over 1 year ago
If it's not in core, can we still use it in net-http
and other gems like that?
Updated by Eregon (Benoit Daloze) over 1 year ago
Probably not, but I also think net-http shouldn't use it.
There is overhead to e.g. call Addrinfo#inspect (or #to_s) in your screenshot above.
Also for such logging to be useful one most likely needs string interpolation, which means string allocations even when the annotations are not used.
And non-Fiber users probably have little to no use for those annotations.
Updated by ioquatix (Samuel Williams) over 1 year ago
I understand your concerns.
Every production system I've worked on has some kind of APM, and I've never heard anyone complain about the overhead - it's usually minuscule in comparison to the actual work being done.
I'm fine for it to be a gem, but it will limit the application to core libraries. Careful application of such a feature can be incredibly helpful for debugging.
One way to mitigate the performance is to use things like prepend
to inject annotations to existing code. But it's clunky at best.
Updated by hsbt (Hiroshi SHIBATA) 8 months ago
- Status changed from Open to Assigned
Updated by ioquatix (Samuel Williams) 8 months ago
- Status changed from Assigned to Closed
I released this as a gem.
https://github.com/ioquatix/fiber-annotation
Maybe it can be adopted if it proves itself to be useful.