Bug #21373: "Illegal instruction: 4" when raising SystemStackError after rescuing

• Status: Open
• Assignee: -
• Target version: -
• ruby -v: ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [x86_64-darwin22]
• Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN

[Ethan (Ethan -)]

I overflowed the stack in a test - this was rescued and tests continued, but when the next test also overflowed the stack, instead of handling the exception, ruby exited with "Illegal instruction: 4", exit code 132, and no other output (no stack trace or dump).

A minimal reproduction follows. The C method JSON::Ext::Generator::GeneratorMethods::Hash#to_json alternates with X#to_json on the stack; this does not reproduce without the C method.

This is on macos and does not reproduce on linux.

require 'json'
class X
  def to_json(*args)
    {'data' => self}.to_json(*args)
  end
end

# raise and rescue once works fine
begin
  JSON.dump(X.new)
rescue SystemStackError
end

# upon overflowing stack a second time, ruby exits abnormally
# with "Illegal instruction: 4" instead of SystemStackError
JSON.dump(X.new)

[mame (Yusuke Endoh)]

• Is duplicate of Bug #18903: Stack overflow signal handling seems to be triggered once and then not working after added

[Ethan (Ethan -)]

Ah, I missed that one. Can close this duplicate, I think.