Project

General

Profile

Actions

Bug #9683

closed

Segmentation fault when using default proc feature in ruby Hash object

Added by levieb (Eran Barak Levi) about 10 years ago. Updated over 4 years ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
ruby 2.1.0p0 (2013-12-25 revision 44422) [x86_64-darwin12.0]
[ruby-core:61720]

Description

➜ bruno-v2 rvm:(ruby-2.1.0) git:(master) ✗ uname -a
Darwin erans-mbp.intkontera.com 13.1.0 Darwin Kernel Version 13.1.0: Thu Jan 16 19:40:37 PST 2014; root:xnu-2422.90.20~2/RELEASE_X86_64 x86_64
➜ bruno-v2 rvm:(ruby-2.1.0) git:(master) ✗ rvm -v
rvm 1.25.19 (stable) by Wayne E. Seguin , Michal Papis [https://rvm.io/]
➜ bruno-v2 rvm:(ruby-2.1.0) git:(master) ✗ ruby -v
ruby 2.1.0p0 (2013-12-25 revision 44422) [x86_64-darwin12.0]
➜ bruno-v2 rvm:(ruby-2.1.0) git:(master) ✗ ruby -e "a = Hash.new {|h,k| h[k] += 1};a[1]" 2> output
[1] 24934 segmentation fault ruby -e "a = Hash.new {|h,k| h[k] += 1};a[1]" 2> output

more information in the attached files


Files

ruby_2014-03-27-153720_Erans-MacBook-Pro.crash (45.3 KB) ruby_2014-03-27-153720_Erans-MacBook-Pro.crash crash report levieb (Eran Barak Levi), 03/27/2014 01:50 PM
output (489 KB) output script output levieb (Eran Barak Levi), 03/27/2014 01:50 PM
9683.patch (1.31 KB) 9683.patch srawlins (Sam Rawlins), 04/01/2014 06:42 PM

Updated by levieb (Eran Barak Levi) about 10 years ago

In ruby-2.0.0
➜ bruno-v2 rvm:(ruby-2.0.0) git:(master) ✗ ruby -v
ruby 2.0.0p451 (2014-02-24 revision 45167) [x86_64-darwin13.1.0]
➜ bruno-v2 rvm:(ruby-2.0.0) git:(master) ✗ irb
2.0.0-p451 :001 > a = Hash.new {|h,k| h[k] += 1}
=> {}
2.0.0-p451 :002 > a[1]
SystemStackError: stack level too deep
from /Users/eran/.rvm/rubies/ruby-2.0.0-p451/lib/ruby/2.0.0/irb/workspace.rb:86
Maybe IRB bug!
2.0.0-p451 :003 > a[1]
[1] 25208 segmentation fault irb

Updated by srawlins (Sam Rawlins) almost 10 years ago

Currently, trunk act's like Eran's example: a[1] first results in SystemStackError, then results in segmentation fault. I've tracked the segfault down to vm.c, in vm_exec():

1327  vm_loop_start:
1328      result = vm_exec_core(th, initial);
1329      if ((state = th->state) != 0) {

I do see that we get to line 1328, but I don't think we actually enter vm_exec_core(). My debugging abilities end there :(

In any case, maybe this should be solved like #9151 : cut it off and return nil if recursion is detected. I've attached a small patch with tests.

This patch prevents the simple SystemStackError case, and permits some legitimate recursion, like in Psych's yaml_tree.rb:

@dispatch_cache = Hash.new do |h,klass|
  method = "visit_#{(klass.name || '').split('::').join('_')}"
  method = respond_to?(method) ? method : h[klass.superclass]
  raise(TypeError, "Can't dump #{target.class}") unless method
  h[klass] = method
end

It also permits creative recursive uses of the default block:

fac = Hash.new {|h,k| h[k] = h[k-1]*k }; fac[1] = 1
fac[10]  #=> 3628800 == 10 factorial
fib = Hash.new {|h,k| h[k] = h[k-2] + h[k-1] }; fib[1] = fib[2] = 1
fib[10]  #=> 55, the 10th Fibonacci number
collatz = Hash.new {|h,k| h[k] = k.even? ? h[k/2] + 1 : h[3*k+1] + 1 }
collatz[1] = collatz[2] = collatz[4] = 0
collatz[7]  #=> 14, the # of Collatz sequence steps to get from 7 to the 1-4-2 loop

Updated by nobu (Nobuyoshi Nakada) almost 10 years ago

I could reproduce it with 2.0.0p353 but 2.0.0p466 occurred SystemStackError.

Updated by srawlins (Sam Rawlins) almost 10 years ago

Nobu, Is p466 an unreleased version? Is it ruby_2_0_0 branch? In any case, you are correct: can reproduce in 2.0.0-p451 and -353. 2.1.x seem to correctly SystemStackError.

There remains a bug in trunk (and 2.1.x), however: the first SystemStackError seems to leave Ruby (or the thread) in a dirty state, if it is rescued. Similar to Eran's irb example above:

$ cat 9683.rb
a = Hash.new {|h,k| h[k] += 1}
begin; a[1]; rescue SystemStackError; end
a[1]
$ ruby 9683.rb
Segmentation fault

This bug doesn't require reusing a:

$ cat 9683b.rb
a = Hash.new {|h,k| h[k] += 1}
begin; a[1]; rescue SystemStackError; end
b = Hash.new {|h,k| h[k] += 1}
b[1]
$ ruby 9683b.rb
Segmentation fault

But it does seem to require the same cause of the Segmentation fault?

$ cat 9683c.rb 
a = Hash.new {|h,k| h[k] += 1}
def foo; foo end

begin; foo; rescue SystemStackError; end
begin; a[1]; rescue SystemStackError; end
foo
$ ruby 9683c.rb
9683c.rb:2: stack level too deep (SystemStackError)

And the bug only occurs on the same thread:

$ cat 9683d.rb 
a = Hash.new {|h,k| h[k] += 1}
begin
a[1]
rescue SystemStackError
end
t = Thread.new { a[1] }
t.join
$ ruby 9683d.rb
9683d.rb:7: stack level too deep (SystemStackError)

(These examples all used ruby trunk.) My patch above basically ignores whatever problem is occurring by avoiding the SystemStackError altogether. Maybe the easy solution, if not the correct one.

Updated by nobu (Nobuyoshi Nakada) almost 10 years ago

There are two stacks, VM stack and machine stack.

def foo; foo end

causes VM stack overflow, but recursive default proc causes machine stack overflow.
The former is detected by the VM, so no dirty pages is used.

Your patch prevents infinite recursion before it actually occurs, and would be more effective.
However, there can be other recursion calls, I'd want the way to fix them before applying it.

Updated by levieb (Eran Barak Levi) over 9 years ago

Hi guys,
FYI it's reproducible on 2.1.3...
➜ ~ rvm:(ruby-2.1.3) git:(master) irb -v
irb 0.9.6(09/06/30)
➜ ~ rvm:(ruby-2.1.3) git:(master) irb
2.1.3 :001 > a = Hash.new {|h,k| h[k] += 1}
=> {}
2.1.3 :002 > a[1]
SystemStackError: stack level too deep
from /Users/eran/.rvm/rubies/ruby-2.1.3/lib/ruby/2.1.0/irb/workspace.rb:86
Maybe IRB bug!
2.1.3 :003 > a[1]
[1] 64348 segmentation fault irb
➜ ~ rvm:(ruby-2.1.3) git:(master) ruby -v
ruby 2.1.3p242 (2014-09-19 revision 47630) [x86_64-darwin14.0]
➜ ~ rvm:(ruby-2.1.3) git:(master) rvm -v
rvm 1.25.33 (stable) by Wayne E. Seguin , Michal Papis [https://rvm.io/]

Updated by hsbt (Hiroshi SHIBATA) almost 8 years ago

  • Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN to 2.2: REQUIRED, 2.3: DONTNEED, 2.4: DONTNEED

I tried to following code with latest stable versions.

a = Hash.new {|h,k| h[k] += 1}
begin; a[1]; rescue SystemStackError; end
a[1]
ruby 2.1.10p492 (2016-04-22 revision 54691) [x86_64-darwin15.0]

ruby 2.2.6p320 (2016-04-26 revision 54784) [x86_64-darwin15]
9683.rb:1:in `yield': stack level too deep (SystemStackError)
        from 9683.rb:1:in `block in <main>'
        from 9683.rb:1:in `yield'
        from 9683.rb:1:in `block in <main>'
        from 9683.rb:1:in `yield'
        from 9683.rb:1:in `block in <main>'
        from 9683.rb:1:in `yield'
        from 9683.rb:1:in `block in <main>'
        from 9683.rb:1:in `yield'
         ... 8719 levels...
        from 9683.rb:1:in `yield'
        from 9683.rb:1:in `block in <main>'
        from 9683.rb:2:in `yield'
        from 9683.rb:2:in `<main>'
ruby 2.3.2p118 (2016-05-06 revision 54924) [x86_64-darwin15]

ruby 2.4.0dev (2016-05-06 trunk 54925) [x86_64-darwin15]

I got SystemStackError only Ruby 2.2 . Does Anyone know related commits?

Actions #8

Updated by jeremyevans0 (Jeremy Evans) over 4 years ago

  • Status changed from Open to Closed
Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0