Feature #19326
openPlease add a better API for passing a Proc to a Ractor
Description
Example 1:
class Worker
def initialize(&block)
@block = block
end
def run
Ractor.new(@block, &:call)
end
end
worker = Worker.new { 1 }
puts worker.run.take
Errors with:
<internal:ractor>:271:in `new': allocator undefined for Proc (TypeError)
from scripts/run.rb:9:in `run'
from scripts/run.rb:14:in `<main>'
Example 2:
class Worker
def initialize(&block)
@block = Ractor.make_shareable(block)
end
def run
Ractor.new(@block, &:call)
end
end
worker = Worker.new { 1 }
puts worker.run.take
Errors with:
<internal:ractor>:820:in `make_shareable': Proc's self is not shareable: #<Proc:0x00007f00394c38b8 scripts/run.rb:13> (Ractor::IsolationError)
from scripts/run.rb:5:in `initialize'
from scripts/run.rb:13:in `new'
from scripts/run.rb:13:in `<main>'
Example 3:
class Worker
def initialize(&block)
@block = Ractor.make_shareable(block)
end
def run
Ractor.new(@block, &:call)
end
end
worker = Ractor.current.instance_eval { Worker.new { 1 } }
puts worker.run.take
Works, but having Ractor.current.instance_eval
as a wrapper around the block is not ideal, as Ractor is supposed to be only an implementation detail in Worker.
I know about https://bugs.ruby-lang.org/issues/18243 and the discussion around proc.bind(nil)
. That would actually be ideal, as for the purposes if why I want this functionality I don't care what self
is in a block, and the less it has access to the better.
The general idea of Worker is to have a Ractor be able to lazily execute an arbitrary proc. And all the bindings it would need would be passed explicitly, either through args
in the constructor or through send
/receive
, so self
would really not matter.
The benefit: this would make it so concurrent code can be more easily be implemented with Ractors as currently you can execute an arbitrary proc by passing it to a Thread (but you don't get the nice data isolation).
Updated by luke-gru (Luke Gruber) almost 2 years ago
If you want Ractor to be an implementation detail of Worker you could do:
class Worker
def initialize(&block)
@block = block
end
def run
block = @block
true.instance_eval { Ractor.new(&block) }
end
end
worker = Worker.new { 1 }
puts worker.run.take
Just an idea :)
It is strange that self
is checked even when Ractor.new
is given a block not yet marked shareable, because the given block's self is changed to the current ractor during execution. Ractor.make_shareable()
, however, makes sense to check self
.
Updated by sdwolfz (Codruț Gușoi) almost 2 years ago
OK, I had absolutely no idea you could do that. Thanks for the suggestion! My impressions was instance_eval
could only be used around where a block is written.
Can we document this behaviour somewhere? Maybe in https://docs.ruby-lang.org/en/3.2/Ractor.html right after the Shareable and unshareable objects
section. I suspect many more people will hit the same wall as I had and would not have an easy time figuring it out.
Updated by Eregon (Benoit Daloze) almost 2 years ago
I'm not sure this behavior is intended, cc @ko1 (Koichi Sasada).
I thought Ractor.new
would only accept literal blocks.
The problem is the usual "does not respect the intent of the writer of the block, which expected a specific self".
Also it only works if you are fine to create a Ractor per block, which seems a big overhead.
Maybe Proc#isolate(new_self) or so should be exposed.
OTOH Ractor is incompatible with most gems out there, so seems of limited usefulness.
For true parallelism compatible with existing Ruby code, use threads on TruffleRuby/JRuby.
Updated by sdwolfz (Codruț Gușoi) almost 2 years ago
I thought Ractor.new would only accept literal blocks.
I hope not, passing an arbitrary block helps with building functionality on top of Ractors (that I want to do).
The problem is the usual "does not respect the intent of the writer of the block, which expected a specific self".
My "Worker" class will have documentation stating that self is replaced and all dependencies need to be passed through the constructor so that's not going to be an issue.
Also it only works if you are fine to create a Ractor per block, which seems a big overhead.
I could have a "Ractor pool" basically limiting the number of Ractors that can run at the same time. Still, the overhead could be improved with future Ruby releases (just like it is for Erlang/Elixir). For all practical purposes so far the overhead is negligible compared to the work done in the blocks.
Maybe Proc#isolate(new_self) or so should be exposed.
This would be a nice functionality to have, indeed.
OTOH Ractor is incompatible with most gems out there, so seems of limited usefulness.
It will become better when Ractor is stabilised, right now I have all I need working.
For true parallelism compatible with existing Ruby code, use threads on TruffleRuby/JRuby.
I'd rather use the real Ruby and stay away from Oracle.
All that being said, I'm really interested to see how this functionality evolves.
Updated by sdwolfz (Codruț Gușoi) almost 2 years ago
New example:
class Worker
def initialize(&block)
@block = block
end
def start
block = Ractor.make_shareable(@block)
@ractor = Ractor.new(block) do |callable|
message = receive
callable.call(message)
end
self
end
def work(*args)
@ractor.send(args)
@ractor.take
end
end
worker = Ractor.current.instance_eval do
Worker.new { |args| puts args.inspect }
end
worker.start.work(1, 2, 3)
Say I don't want to just pass the bock as is to the Ractor, but I want to do some setup first within the Ractor itself. In that case I'm back to needing both make_shareable
and the instance_eval
around the literal block.
Updated by hsbt (Hiroshi SHIBATA) almost 2 years ago
- Status changed from Open to Assigned
- Assignee set to ko1 (Koichi Sasada)
Updated by luke-gru (Luke Gruber) almost 2 years ago
Your new example, I think, is not possible with the current API.
However, there is a bug in Ruby that I just found that allows it.
class Worker
def start(&blk)
blk = blk.curry # bug in ruby allows sharing of non-shareable proc
Ractor.make_shareable(blk)
@ractor = Ractor.new(blk) do |b|
message = receive
b.call(message)
end
self
end
def work(msg)
@ractor.send(msg)
@ractor.take
end
end
worker = Worker.new
worker.start { |args| p args }
worker.work('msg')
This is a bug because in debug versions of ruby this crashes. I opened a ticket for it: https://bugs.ruby-lang.org/issues/19374
Updated by forthoney (Seong-Heon Jung) 11 months ago
Are there any updates on this issue?