Feature #20876
Updated by ioquatix (Samuel Williams) 15 days ago
This is an evolution of the previous proposal: https://bugs.ruby-lang.org/issues/20855
## Background
The current Fiber Scheduler performance can be significantly impacted by blocking operations that cannot be deferred to the event loop, particularly in high-concurrency environments where Fibers rely on non-blocking operations for efficient task execution.
## Proposal
Pull Request: https://github.com/ruby/ruby/pull/12016
We will introduce a new fiber scheduler hook called `blocking_operation_work`:
```ruby
class MySchduler
# ...
def blocking_operation_wait(work)
# Example (trivial) implementation:
Thread.new(&work).join
end
end
```
We introduce a new flag for `rb_nogvl`: `RB_NOGVL_OFFLOAD_SAFE` which indicates that `rb_nogvl(func, ...)` is a blocking operation that is safe to execute on a different thread or thread pool (or some other context).
When a C extension invokes `rb_nogvl(..., RB_NOGVL_OFFLOAD_SAFE)`, and a fiber scheduler is available, all the arguments will be saved into a instance of a callable object (at this time a `Proc`) called `work` and passed to the `blocking_operation_wait` fiber scheduler hook. When `work` is `#call`ed, it will execute `rb_nogvl` again with all the same arguments.
The fiber scheduler can decide how to execute that work, e.g. on a separate thread or thread pool, to mitigate the performance impact of the blocking operation on the event loop.
### Cancellation
`rb_nogvl` takes several arguments, a `func` for the actual work, and `unblock_func` to cancel `func` if possible. These arguments are preserved in the `work` proc, and cancellation works the same. However, some extra effort may be required in the fiber scheduler hook, e.g.
```ruby
class MySchduler
# ...
def blocking_operation_wait(work)
thread = Thread.new(&work)
thread.join
thread = nil
ensure
thread&.kill
end
end
```
### Interruption Points
When using the `RB_NOGVL_OFFLOAD_SAFE` flag, the semantics of interruption points in `rb_nogvl` changes.
Currently, by default, `rb_nogvl` only checks for interrupts **after** executing the `BLOCKING_REGION`. However, when using `RB_NOGVL_OFFLOAD_SAFE`, an interruption point is introduced **before** executing the `BLOCKING_REGION` beause we invoke Ruby code (the fiber scheduler hook) before the blocking operation is performed.
## Example
Using the branch of `async` gem: https://github.com/socketry/async/pull/352/files and enabling zlib deflate to use this feature, the following performance improvement was achieved:
```ruby
require "zlib"
require "async"
require "benchmark"
DATA = Random.new.bytes(1024*1024*100)
duration = Benchmark.measure do
Async do
10.times do
Async do
Zlib.deflate(DATA)
end
end
end
end
# Ruby 3.3.4: ~16 seconds
# Ruby 3.4.0 + PR: ~2 seconds.
```
To run this benchmark yourself, you must compile CRuby with these two PRs:
- https://github.com/ruby/ruby/pull/12016
- https://github.com/ruby/zlib/pull/88
In addition, enable `RB_NOGVL_OFFLOAD_SAFE` in `zlib.c`'s call to `rb_nogvl`.
Then, use this branch of async: https://github.com/socketry/async/pull/352