Feature #18004: Add Async to the stdlib

• Status: Open
• Assignee: -
• Target version: -

[shan (Shannon Skipper)]

Adding Async to the stdlib would signal a clear concurrency story for Ruby 3 to compliment Ractor-based parallelism. I don't know how ioquatix feels about adding Async to stdlib, but I wanted to propose it since we keep getting questions about concurrent I/O with Ruby 3 in the community.

Ractors get a fair amount of attention on the #ruby IRC channels and Ruby Discord. When Ractors are discussed, question around concurrent I/O in Ruby 3 often follow. Folk don't seem to be aware of Async, so we often cite the Ruby 3 release notes Async Net::HTTP example shown below.

require 'async'
require 'net/http'
require 'uri'

Async do
  ["ruby", "rails", "async"].each do |topic|
    Async do
      Net::HTTP.get(URI "https://www.google.com/search?q=#{topic}")
    end
  end
end

The main downside I see for this proposal is the bloat from Async's several gem dependencies. For what it's worth, nio4r has been a staple for a long time and is also the only dependency of Puma.

Async is a composable asynchronous I/O framework for Ruby based on nio4r and timers.

Async is just so useful it would be awesome to add to the stdlib. It fills and important gap for concurrent I/O with Ruby 3 and would be exciting to see included in a future release.

See https://github.com/socketry/async#readme

[shyouhei (Shyouhei Urabe)]

Hello. Async is an awesome gem that I basically support. But there are whole lot of other gems that are equally awesome. What do you think makes async gem so special among other gems that it would be worth being added into the core? These days you need a really decent reason to turn a gem into a standard library.

[shan (Shannon Skipper)]

Hello. Async is an awesome gem that I basically support. But there are whole lot of other gems that are equally awesome. What do you think makes async gem so special among other gems that it would be worth being added into the core? These days you need a really decent reason to turn a gem into a standard library.

Hi! Concurrent I/O seems like it is becoming more of a core consideration for languages these days, similar to how parallel processing has become increasingly important to get right. I felt like Ractors gave a good path away from Threads for parallel processing but there isn't a clear alternative for concurrency. It seemed nice for Ruby to also ship with a concurrent I/O library if a preferred one can be adopted. I do totally understand and agree that adding a gem to stdlib requires a high bar and exceptional circumstance. Repeated questions from Rubyists in the community about whether Ruby 3 has concurrent I/O just made me wonder if Async is worth promoting to the forefront.

[konsolebox (K B)]

shan (Shannon Skipper) wrote in #note-2:

It seemed nice for Ruby to also ship with a concurrent I/O library if a preferred one can be adopted.

Or implement Promises plus await/async in Ruby maybe? The async and await combo especially is very powerful at simplifying asynchronous code. They are more than just a syntactic sugar.

[larskanis (Lars Kanis)]

Adding the core Async gem to stdlib requires to add 4 more dependent gems. Async is a big library with a lot of supplemental gems. It makes things more complicated if some gems are in stdlib and some are not. Also Async is made to be usable on rubies before 3.0, whereas ruby-3.x has Fiber.schedule as a builtin mechanism for starting async tasks.

The problem is, that Fiber.schedule is not usable without adding a complete scheduler. This makes Fiber.schedule a no-go in smaller scripts. So IMHO the thing that is missing in ruby-3.x is a simple scheduler. Otherwise the scheduler feature is rather incomplete.

My proposal is to promote test/fiber/scheduler.rb to the stdlib (probably as a gem). With around 250 lines it is small enough to be considered a simple reference scheduler, but is too big to be copied into each and every script. Would you consider such a pull request?

Background: I'm going to make ruby-pg fully compatible to Fiber.scheduler here. For testing a simple scheduler is required, so I copied test/fiber/scheduler.rb, but it doesn't receive fixes like this when copied.

[hsbt (Hiroshi SHIBATA)]

I'm against this. We don't want to add the new dependency for stdlib.

[shan (Shannon Skipper)]

larskanis (Lars Kanis) wrote in #note-4:

Adding the core Async gem to stdlib requires to add 4 more dependent gems. Async is a big library with a lot of supplemental gems. It makes things more complicated if some gems are in stdlib and some are not. Also Async is made to be usable on rubies before 3.0, whereas ruby-3.x has Fiber.schedule as a builtin mechanism for starting async tasks.

The problem is, that Fiber.schedule is not usable without adding a complete scheduler. This makes Fiber.schedule a no-go in smaller scripts. So IMHO the thing that is missing in ruby-3.x is a simple scheduler. Otherwise the scheduler feature is rather incomplete.

My proposal is to promote test/fiber/scheduler.rb to the stdlib (probably as a gem). With around 250 lines it is small enough to be considered a simple reference scheduler, but is too big to be copied into each and every script. Would you consider such a pull request?

Background: I'm going to make ruby-pg fully compatible to Fiber.scheduler here. For testing a simple scheduler is required, so I copied test/fiber/scheduler.rb, but it doesn't receive fixes like this when copied.

That's a great point about multiple gem dependencies. I think my request was overly broad and you're right that a tiny but maintained scheduler backed by io-wait would be great to have by default rather than the whole Async gem along with it's dependencies.

It looks like https://github.com/bruno-/fiber_scheduler would be a good option, though it could be paired down by removing scheduler.rb since io-wait is guaranteed to be available.

[shan (Shannon Skipper)]

Is it better to retitle this issue to align with larskanis's proposal or close this and start a new issue? Happy to do either!

[shan (Shannon Skipper)]

Or I can just close it and be patient, since I assume a scheduler will be added eventually. :)

It really would be nice to have a minimalist scheduler at some point, since the test implementations in ruby/ruby and ruby/io-wait aren't the easiest to use and it's an incredibly useful feature that's not easy to implement yourself and just this last little bit blocks Fiber.schedule use.