Project

General

Profile

Bug #13164

A second `SystemStackError` exception results in `Segmentation fault (core dumped)`

Added by myst (Boaz Segev) over 2 years ago. Updated over 2 years ago.

Status:
Open
Priority:
Normal
Assignee:
-
Target version:
-
ruby -v:
ruby 2.4.0p0 (2016-12-24 revision 57164) [x86_64-darwin16]
[ruby-core:79285]

Description

This issue is was exposed by leveraging the fact that Object#hash is implemented recursively for core Ruby datatypes (i.e., Hash and Array). See the discussion here: https://github.com/boazsegev/combine_pdf/pull/91#issuecomment-275552131.

TO reproduce the issue, explode the stack twice.

Expected results:

SystemStackError will be raised both times.

Actual results:

SystemStackError is raised once. The second time will cause a core dump.

Code to cause core dump:

def compute_nest_depth
  h = {nest: {}}
  nest = h[:nest]
  i = 0

  while true
    i += 1
    puts "nested #{i}" if ((i & 511) == 0)
    next_nest = { nest: {} }
    nest[:nest] = next_nest
    nest = next_nest[:nest]
    h.hash
  end

rescue SystemStackError
  puts "Stack exploded at nesting #{i}"
end

counter = 0;
while(true)
  begin
    counter +=1
    puts "starting test #{counter}"
    compute_nest_depth
  rescue SystemStackError => e
    nil
  ensure
    puts "test #{counter} complete"
  end
end

results:

starting test 1
nested 512
nested 1024
nested 1536
nested 2048
nested 2560
Stack exploded at nesting 2783
test 1 complete
starting test 2
nested 512
nested 1024
nested 1536
nested 2048
nested 2560
Segmentation fault (core dumped)

Related issues

Related to Ruby master - Bug #13412: Infinite recursion with define_method may cause silent SEGV or cfp consistency errorClosedActions
Related to Ruby master - Bug #13948: Segfault instead of recursion depth error ClosedActions
Has duplicate Ruby master - Bug #13596: Segfault when catching SystemStackError in evalClosedActions

Associated revisions

Revision 58352
Added by nobu (Nobuyoshi Nakada) over 2 years ago

configure.in: sigsetjmp sivesigs flag

  • configure.in (RUBY_SETJMP_TYPE): optional flag to save signal mask.

Revision 1e1a5853
Added by nobu (Nobuyoshi Nakada) over 2 years ago

signal.c: unblock signal

  • signal.c (raise_stack_overflow): unblock the received signal, to receive the same signal again. [ruby-core:79285] [Bug #13164]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58353 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 58353
Added by nobu (Nobuyoshi Nakada) over 2 years ago

signal.c: unblock signal

  • signal.c (raise_stack_overflow): unblock the received signal, to receive the same signal again. [ruby-core:79285] [Bug #13164]

Revision 58353
Added by nobu (Nobuyoshi Nakada) over 2 years ago

signal.c: unblock signal

  • signal.c (raise_stack_overflow): unblock the received signal, to receive the same signal again. [ruby-core:79285] [Bug #13164]

Revision 58353
Added by nobu (Nobuyoshi Nakada) over 2 years ago

signal.c: unblock signal

  • signal.c (raise_stack_overflow): unblock the received signal, to receive the same signal again. [ruby-core:79285] [Bug #13164]

Revision 58361
Added by nobu (Nobuyoshi Nakada) over 2 years ago

signal.c: prefer pthread_sigmask

  • signal.c (raise_stack_overflow): prefer pthread_sigmask to sigprocmask, for multithreading.

History

Updated by nobu (Nobuyoshi Nakada) over 2 years ago

By doubling rb_sigaltstack_size(), it doesn't segfault and the second or more stack overflows never happen now.
I suspect that the stack guard page may need to be reset, but not sure.

diff --git a/signal.c b/signal.c
index 888c8eaa72..8947b0ea95 100644
--- a/signal.c
+++ b/signal.c
@@ -563,7 +563,7 @@ rb_sigaltstack_size(void)
     }
 #endif

-    return size;
+    return size * 2;
 }

 /* alternate stack for SIGSEGV */

Updated by myst (Boaz Segev) over 2 years ago

This is a good observation and I'm happy you found this...

However, I'm not sure that using return size * 2 as a patch will solve the issue. It might end up masking the real issue, making it harder to find (although I might be wrong).

At the moment, there is a segmentation fault. Is it possible that the size returned is somehow effecting a memory address / pointer in a way that it shouldn't...?

Updated by nobu (Nobuyoshi Nakada) over 2 years ago

When configured with --with-setjmp-type=sigsetjmp, it seemed working.
But segfaulted at the fourth system stack overflow.

Updated by myst (Boaz Segev) over 2 years ago

What about flattening recursion in core types (Hash, Array and Set)?

I know this won't resolve the issue, but it will prevent eql? and hash from exploding the stack, so the issue is less likely to occur when there isn't an error in the code being executed.

#5

Updated by nobu (Nobuyoshi Nakada) over 2 years ago

  • Has duplicate Bug #13412: Infinite recursion with define_method may cause silent SEGV or cfp consistency error added
#6

Updated by nobu (Nobuyoshi Nakada) over 2 years ago

  • Has duplicate deleted (Bug #13412: Infinite recursion with define_method may cause silent SEGV or cfp consistency error)
#7

Updated by nobu (Nobuyoshi Nakada) over 2 years ago

  • Related to Bug #13412: Infinite recursion with define_method may cause silent SEGV or cfp consistency error added
#8

Updated by nobu (Nobuyoshi Nakada) over 2 years ago

  • Status changed from Open to Closed

Applied in changeset trunk|r58353.


signal.c: unblock signal

  • signal.c (raise_stack_overflow): unblock the received signal, to receive the same signal again. [ruby-core:79285] [Bug #13164]

Updated by nobu (Nobuyoshi Nakada) over 2 years ago

  • Status changed from Closed to Open

On Linux, fixed by unblocking the received signal.
But it has no effect on mac OS and seems to need --with-setjmp-type=setjmp.

#10

Updated by nobu (Nobuyoshi Nakada) over 2 years ago

  • Has duplicate Bug #13596: Segfault when catching SystemStackError in eval added
#11

Updated by shyouhei (Shyouhei Urabe) almost 2 years ago

  • Related to Bug #13948: Segfault instead of recursion depth error added

Also available in: Atom PDF