Bug #15599: Mixing autoload and require causes deadlock and incomplete definition.

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 2.7.0dev (2019-02-11 trunk 67049) [x86_64-linux]
• Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN
• Tags: core, autoload

[akr (Akira Tanaka)]

I found that mixing autoload and require causes deadlock and incomplete definition.

% cat a.rb 
class A
  def a1() end
end
% cat base.rb 
autoload :A, './a'

t1 = Thread.new { p A.instance_methods(false) }
t2 = Thread.new { require './a' }

t1.join
t2.join
% ruby base.rb 
Traceback (most recent call last):
	1: from base.rb:6:in `<main>'
base.rb:6:in `join': No live threads left. Deadlock? (fatal)
3 threads, 3 sleeps current:0x000055cc6943fde0 main thread:0x000055cc6935f4b0
* #<Thread:0x000055cc6938f190 sleep_forever>
   rb_thread_t:0x000055cc6935f4b0 native:0x00007f58d256eb40 int:0
   base.rb:6:in `join'
   base.rb:6:in `<main>'
* #<Thread:[e-mail]:3 sleep_forever>
   rb_thread_t:0x000055cc69736180 native:0x00007f58ce7b3700 int:0 mutex:0x000055cc6943fde0 cond:1
    depended by: tb_thread_id:0x000055cc6935f4b0
   /home/akr/ruby/o0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:54:in `require'
   /home/akr/ruby/o0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:54:in `require'
   base.rb:3:in `block in <main>'
* #<Thread:[e-mail]:4 sleep_forever>
   rb_thread_t:0x000055cc6943fde0 native:0x00007f58ce5b1700 int:0
   /tmp/a/a.rb:1:in `<top (required)>'
   /home/akr/ruby/o0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:54:in `require'
   /home/akr/ruby/o0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:54:in `require'
   base.rb:4:in `block in <main>'
% ruby base.rb
[:a1]
% ruby base.rb
[]

The last run which prints [] means incomplete definition of A which a1 method is not defined.

[akr (Akira Tanaka)]

• Related to Bug #15598: Deadlock on mutual reference of autoloaded constants added

[akr (Akira Tanaka)]

• Related to Misc #21154: Document or change Module#autoload? added

[wanabe (_ wanabe)]

• Status changed from Open to Closed

This issue was fixed between preview 2 and preview 3 of Ruby 3.3.0.

$ cat a.rb
class A
  def a1() end
end
$ cat base.rb 
1000.times do
  fork do
    autoload :A, './a'

    t1 = Thread.new { raise if A.instance_methods(false).size != 1 }
    t2 = Thread.new { require './a' }

    begin
      t1.join
      t2.join
    rescue Exception => e
      exit 1
    end
  end
end
p Process.waitall.map {|_pid, status| status.success? ? :success : :failure }.group_by(&:itself).transform_values(&:size).to_a
puts :fin
$ docker run -it --rm -v.:/work -w/work --env ALL_RUBY_SINCE=3.2.10 rubylang/all-ruby /all-ruby/all-ruby base.rb
ruby-3.2.10         #<Thread:0x00007119bd008e88 base.rb:5 run> terminated with exception (report_on_exception is true):
                    base.rb:5:in `block (3 levels) in <main>': unhandled exception
                    #<Thread:0x00007119bd008e88 base.rb:5 run> terminated with exception (report_on_exception is true):
                    base.rb:5:in `block (3 levels) in <main>': unhandled exception
                    #<Thread:0x00007119bd008e88 base.rb:5 run> terminated with exception (report_on_exception is true):
                    base.rb:5:in `block (3 levels) in <main>': unhandled exception
                    #<Thread:0x00007119bd008e88 base.rb:5 run> terminated with exception (report_on_exception is true):
                    base.rb:5:in `block (3 levels) in <main>': unhandled exception
                    [[:success, 921], [:failure, 79]]
                    fin
ruby-3.2.11         #<Thread:0x00007eb67a448f68 base.rb:5 run> terminated with exception (report_on_exception is true):
                    base.rb:5:in `block (3 levels) in <main>': unhandled exception
                    [[:success, 922], [:failure, 78]]
                    fin
ruby-3.3.0-preview1 #<Thread:0x00007429d36e67e0 base.rb:5 run> terminated with exception (report_on_exception is true):
                    base.rb:5:in `block (3 levels) in <main>': unhandled exception
                    [[:success, 931], [:failure, 69]]
                    fin
ruby-3.3.0-preview2 #<Thread:0x0000751915115f78 base.rb:5 run> terminated with exception (report_on_exception is true):
                    base.rb:5:in `block (3 levels) in <main>': unhandled exception
                    #<Thread:0x0000751915115f78 base.rb:5 run> terminated with exception (report_on_exception is true):
                    base.rb:5:in `block (3 levels) in <main>': unhandled exception
                    #<Thread:0x0000751915115f78 base.rb:5 run> terminated with exception (report_on_exception is true):
                    base.rb:5:in `block (3 levels) in <main>': unhandled exception
                    #<Thread:0x0000751915115f78 base.rb:5 run> terminated with exception (report_on_exception is true):
                    base.rb:5:in `block (3 levels) in <main>': unhandled exception
                    #<Thread:0x0000751915115f78 base.rb:5 run> terminated with exception (report_on_exception is true):
                    base.rb:5:in `block (3 levels) in <main>': unhandled exception
                    #<Thread:0x0000751915115f78 base.rb:5 run> terminated with exception (report_on_exception is true):
                    base.rb:5:in `block (3 levels) in <main>': unhandled exception
                    #<Thread:0x0000751915115f78 base.rb:5 run> terminated with exception (report_on_exception is true):
                    base.rb:5:in `block (3 levels) in <main>': unhandled exception
                    #<Thread:0x0000751915115f78 base.rb:5 run> terminated with exception (report_on_exception is true):
                    base.rb:5:in `block (3 levels) in <main>': unhandled exception
                    [[:success, 904], [:failure, 96]]
                    fin
ruby-3.3.0-preview3 [[:success, 1000]]
                    fin
...
ruby-4.0.6          [[:success, 1000]]
                    fin

I found the commit is be1bbd5b7d40ad863ab35097765d3754726bbd54 == https://bugs.ruby-lang.org/issues/19842.
I don't know why this is the case, but I suspect that the refactoring done to achieve the original goal (M:N threads) had a positive effect.

$ git checkout be1bbd5b7d40ad863ab35097765d3754726bbd54~ && make -j miniruby >/dev/null 2>&1 && ./miniruby base.rb
HEAD is now at 096ee0648e2 Bump ruby/setup-ruby from 1.155.0 to 1.156.0
[[:success, 945], [:failure, 55]]
fin
$ git checkout be1bbd5b7d40ad863ab35097765d3754726bbd54 && make -j miniruby >/dev/null 2>&1 && ./miniruby base.rb
Previous HEAD position was 096ee0648e2 Bump ruby/setup-ruby from 1.155.0 to 1.156.0
HEAD is now at be1bbd5b7d4 M:N thread scheduler for Ractors
[[:success, 1000]]
fin