Bug #9891: infinite fibers crash Ruby

• Status: Closed
• Assignee: ko1 (Koichi Sasada)
• Target version: 2.2.0
• ruby -v: ruby 2.2.0dev (2014-06-01 trunk 46287) [x86_64-openbsd5.5]
• Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN

[kernigh (George Koehler)]

When a program makes too many fibers, Ruby crashes. I found this bug while playing with an Enumerator that recursively enumerates itself.

Bug is easy to reproduce. This one-line fiber bomb will crash Ruby:

p = proc { Fiber.new(&p).resume }; p.call

I expect that Ruby should not crash, but should raise OutOfMemoryError or SystemStackError or something like that.

$ ruby -e 'p = proc { Fiber.new(&p).resume }; p.call'
[BUG] Segmentation fault at 0x00000000000018
ruby 2.2.0dev (2014-06-01 trunk 46287) [x86_64-openbsd5.5]

-- Control frame information -----------------------------------------------
c:0001 p:---- s:0001 e:000000 ------


-- Other runtime information -----------------------------------------------

* Loaded script: -e

* Loaded features:

    0 enumerator.so
    1 /home/kernigh/prefix/lib/ruby/2.2.0/x86_64-openbsd5.5/enc/encdb.so
    2 /home/kernigh/prefix/lib/ruby/2.2.0/x86_64-openbsd5.5/enc/trans/transdb.so
    3 /home/kernigh/prefix/lib/ruby/2.2.0/x86_64-openbsd5.5/rbconfig.rb
    4 /home/kernigh/prefix/lib/ruby/2.2.0/rubygems/compatibility.rb
    5 /home/kernigh/prefix/lib/ruby/2.2.0/rubygems/defaults.rb
    6 /home/kernigh/prefix/lib/ruby/2.2.0/rubygems/deprecate.rb
    7 /home/kernigh/prefix/lib/ruby/2.2.0/rubygems/errors.rb
    8 /home/kernigh/prefix/lib/ruby/2.2.0/rubygems/version.rb
    9 /home/kernigh/prefix/lib/ruby/2.2.0/rubygems/requirement.rb
   10 /home/kernigh/prefix/lib/ruby/2.2.0/rubygems/platform.rb
   11 /home/kernigh/prefix/lib/ruby/2.2.0/rubygems/basic_specification.rb
   12 /home/kernigh/prefix/lib/ruby/2.2.0/rubygems/stub_specification.rb
   13 /home/kernigh/prefix/lib/ruby/2.2.0/rubygems/util/stringio.rb
   14 /home/kernigh/prefix/lib/ruby/2.2.0/rubygems/specification.rb
   15 /home/kernigh/prefix/lib/ruby/2.2.0/rubygems/exceptions.rb
   16 /home/kernigh/prefix/lib/ruby/2.2.0/rubygems/core_ext/kernel_gem.rb
   17 thread.rb
   18 /home/kernigh/prefix/lib/ruby/2.2.0/x86_64-openbsd5.5/thread.so
   19 /home/kernigh/prefix/lib/ruby/2.2.0/monitor.rb
   20 /home/kernigh/prefix/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb
   21 /home/kernigh/prefix/lib/ruby/2.2.0/rubygems.rb

[NOTE]
You may have encountered a bug in the Ruby interpreter or extension libraries.
Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html

Abort trap (core dumped) 
$ gdb ruby ruby.core
...
#0  0x00001366bcdf921a in kill () at <stdin>:2
2       <stdin>: No such file or directory.
        in <stdin>
(gdb) bt
#0  0x00001366bcdf921a in kill () at <stdin>:2
#1  0x00001366bce5998a in abort () at /usr/src/lib/libc/stdlib/abort.c:70
#2  0x00001364babd4571 in rb_bug_context (ctx=0x1366c9748c90, 
    fmt=0x1364bad0351e "Segmentation fault at %p") at ../ruby/error.c:361
#3  0x00001364baaf611c in sigsegv (sig=Variable "sig" is not available.
) at ../ruby/signal.c:821
#4  <signal handler called>
#5  0x00001364baa21752 in rb_threadptr_tag_jump (th=0x1366c32adc00, st=6)
    at eval_intern.h:162
#6  0x00001364baa25bf1 in rb_exc_raise (mesg=Variable "mesg" is not available.
) at ../ruby/eval.c:571
#7  0x00001364babd543f in rb_raise (exc=21332200143640, fmt=Variable "fmt" is not available.
)
    at ../ruby/error.c:1915
#8  0x00001364babd547f in rb_error_frozen (what=Variable "what" is not available.
) at ../ruby/error.c:2129
#9  0x00001364bab3d117 in rb_ivar_set (obj=21331955219840, id=9617, 
    val=21332204288760) at ../ruby/variable.c:940
#10 0x00001364baa2528f in setup_exception (th=0x1366c32adc00, tag=6, 
    mesg=21331955219840, cause=21332204288760) at ../ruby/eval.c:458
#11 0x00001364baa25bdc in rb_exc_raise (mesg=21331955219840)
    at ../ruby/eval.c:569
#12 0x00001364baa3bfde in ruby_memerror () at ../ruby/gc.c:6030
#13 0x00001364baa3eac6 in objspace_xmalloc (objspace=0x1366c32ad000, 
    size=192816) at ../ruby/gc.c:6229
#14 0x00001364bab8ce45 in fiber_store (next_fib=Variable "next_fib" is not available.
) at ../ruby/cont.c:403
#15 0x00001364bab8e1bf in rb_fiber_start () at ../ruby/cont.c:1461
---Type <return> to continue, or q <return> to quit---
#16 0x00001364baa22812 in ruby_exec_internal (n=Variable "n" is not available.
) at ../ruby/eval.c:252
#17 0x00001364baa246e6 in ruby_run_node (n=Variable "n" is not available.
) at ../ruby/eval.c:317
#18 0x00001364baa20ad2 in main (argc=3, argv=0x7f7ffffcd188)
    at ../ruby/main.c:36
Current language:  auto; currently asm

[shyouhei (Shyouhei Urabe)]

• Status changed from Open to Assigned
• Assignee set to ko1 (Koichi Sasada)

[ko1 (Koichi Sasada)]

Only on OpenBSD?
On my linux and windows (mswin32/64) doesn't cause SEGV.

[kernigh (George Koehler)]

I have only tried on OpenBSD. I updated ruby to
ruby 2.2.0dev (2014-11-29 trunk 48638) [x86_64-openbsd5.6]

OpenBSD doesn't have getcontext()/setcontext(), so I suspect that cont.c defines FIBER_USE_NATIVE to 0. I don't know if FIBER_USE_NATIVE is important.

Some runs raise NoMemoryError, but some runs crash:

$ ruby -e 'p = proc { Fiber.new(&p).resume }; p.call'
-e:1:in `initialize': failed to allocate memory (NoMemoryError)
        from -e:1:in `new'
        from -e:1:in `block in <main>'
$ ruby -e 'p = proc { Fiber.new(&p).resume }; p.call' 
-e:1:in `resume': failed to allocate memory (NoMemoryError)
        from -e:1:in `block in <main>'
$ ruby -e 'p = proc { Fiber.new(&p).resume }; p.call' 
[BUG] Segmentation fault at 0x00000000000018
ruby 2.2.0dev (2014-11-29 trunk 48638) [x86_64-openbsd5.6]
...

[ko1 (Koichi Sasada)]

OpenBSD doesn't have getcontext()/setcontext(), so I suspect that cont.c defines FIBER_USE_NATIVE to 0. I don't know if FIBER_USE_NATIVE is important.

Great. Maybe malloc() is failed and we can't check it enough.

[ko1 (Koichi Sasada)]

The reason of this SEGV is, rb_longjmp without TAG.

void
rb_fiber_start(void)
{
...
    TH_PUSH_TAG(th);
...   /* protected by TAG */
    TH_POP_TAG();

...
    rb_fiber_terminate(fib); // -> NoMemoryError, but not protected by TAG.
}

It is difficult for me.
Nobu, do you have any idea to solve it?

[kernigh (George Koehler)]

Hello again. I reported this bug 3 years ago. I'm happy to report that I can't reproduce this bug with recent Ruby trunk,

$ ruby -v
ruby 2.5.0dev (2017-09-11 trunk 59840) [x86_64-openbsd6.1]

In recent days, I wrote some Ruby scripts to create infinite fibers, then tried to crash Ruby. I got some crashes, but two other bugs caused all the crashes. I reproduced both bugs without using infinite fibers. I reported bug #13875 to Ruby, and then reported a bug in posix_memalign() to OpenBSD (https://marc.info/?l=openbsd-bugs&m=150524047013850&w=2). I pulled Eric Wong's fix for #13875 and made a one-line fix in OpenBSD libc. After this, Ruby stopped crashing.

I ran Ruby with a shell loop like this one:

(ulimit -d 100000; while ruby oom.rb; [ $? -le 1 ]; do :; done)

In OpenBSD, ulimit -d 100000 limits memory allocation to 100_000 kilobytes, so my scripts failed more quickly, without eating too much RAM or swap space. The loop stops if Ruby crashes.

Here are three of my scripts to create infinite fibers:

pr = proc do
  begin
    Fiber.new(&pr).resume
  rescue NoMemoryError
    puts "Rescuing NoMemoryError"
  end
end

pr.call
puts "Done"

pr = proc do
  begin
    re = Fiber.yield(1)
    puts re if re % 100 == 0
    fi = Fiber.new(&pr)
    re = fi.resume
    while re
      re = Fiber.yield(re + 1)
      re = fi.resume(re + 1)
    end
  rescue NoMemoryError
    puts "Rescuing NoMemoryError"
  end
  re = Fiber.yield(1)
  puts re if re % 100 == 0
end

fi = Fiber.new(&pr)
re = fi.resume
while re
  re = fi.resume(re + 1)
end
puts "Done"

# https://wiki.haskell.org/The_Fibonacci_sequence provides this
# Haskell code for a lazy infinite sequence:
#
#   fibs = 0 : 1 : zipWith (+) fibs (tail fibs)
#
# I try to translate it into Ruby.

fibs = Enumerator.new do |y|
  y << 0
  y << 1
  fibs.zip(fibs.lazy.drop(1)) {|a, b| y << (a + b)}
end

fibs.each {|i| puts i}

I now can't crash Ruby with these scripts. Because I can't reproduce this bug, I suggest closing this bug.

[ko1 (Koichi Sasada)]

• Status changed from Assigned to Closed

Thank you for your confirmation.