Bug #22476: rb_profile_frames() called from a signal handler segfaults while vm_make_env_each() is escaping a chain of frames

• Status: Open
• Assignee: Eregon (Benoit Daloze)
• Target version: -
• ruby -v: ruby 4.1.0dev (2026-10-27T12:50:14Z master 32f06ce98c) +PRISM [arm64-darwin25]
• Backport: 3.3: REQUIRED, 3.4: REQUIRED, 4.0: REQUIRED

[Eregon (Benoit Daloze)]

Sampling profilers (like StackProf, Vernier, Datadog) call rb_profile_frames() from a signal handler.
Datadog uses a slight variant of rb_profile_frames() but it does not matter, it reproduces either way.
rb_profile_frames() segfaults with a NULL dereference if the signal arrives while vm_make_env_each() is moving a chain of frames to the heap, e.g. during proc {}, define_method(:foo) {} or Thread.new {} called from a block.

This was first seen in production with the Datadog profiler on Ruby 3.4 with YJIT (background: https://github.com/DataDog/dd-trace-rb/pull/6173):

ddtrace_rb_profile_frames (private_vm_api_access.c:528)
prepare_sample_thread (collectors_stack.c:636)
handle_sampling_signal (collectors_cpu_and_wall_time_worker.c:686)
__kernel_rt_sigreturn
rb_yjit_invalidate_ep_is_bp (yjit/src/invariants.rs:579)
vm_make_env_each (vm.c:1044)
vm_make_env_each (vm.c:969)
vm_make_env_each (vm.c:969)
rb_vm_make_proc_lambda (vm.c:1429)
rb_mod_define_method_with_visibility (proc.c:2260)
rb_mod_define_method (proc.c:2368)
vm_call_cfunc_with_frame (vm_insnhelper.c:3840)

We hadn't seen it before with the Datadog profiler because we were not retrieving the CME for Ruby frames (only the ISEQ). We were getting the CME only for native frames.

The attached script below reproduces it on master using plain rb_profile_frames(), with no gems. It also crashes without YJIT.

Reproduction

ruby rb_profile_frames_signal_crash.rb (MODE=define_method and MODE=thread also crash). The script compiles a small C extension that installs a SIGPROF handler calling rb_profile_frames(), and starts a pthread that sends SIGPROF to the main thread with pthread_kill().

On ruby 4.1.0dev (2026-10-27T12:50:14Z master 32f06ce98c) +PRISM [arm64-darwin25] it crashes within ~0.03-0.25s with back-to-back signals, and within ~1s with INTERVAL_US=1000 (1 kHz sampling):

rb_profile_frames_signal_crash.rb:112: [BUG] Segmentation fault at 0x0000000000000000
ruby 4.1.0dev (2026-10-27T12:50:14Z master 32f06ce98c) +PRISM [arm64-darwin25]

Native backtrace of the crashing thread (from macOS sample):

rb_vm_make_proc_lambda        vm.c:1733
  vm_make_env_each            vm.c:1141
    _sigtramp
      sigprof_handler         (the SIGPROF handler of the reproducer)
        thread_profile_frames vm_backtrace.c:2144
          _sigtramp
            rb_bug_for_fatal_signal error.c:1167
              rb_vm_bugreport vm_dump.c:1433
                rb_vmdebug_stack_dump_raw vm_dump.c:503

Side note: after printing the [BUG] header, the crash reporter hangs in rb_vmdebug_stack_dump_raw() (vm_dump.c:503) and the process never exits, so it needs timeout -s KILL. Probably it trips over the same inconsistent frame.

Cause

Let B be a block frame whose prev EP is a local frame A that has not escaped yet. vm_make_env_each(B):

1. recurses into vm_make_env_each(A) (vm.c:1141). At the end, this overwrites the flags slot of A's on-stack EP with the env object: VM_STACK_ENV_WRITE(ep, 0, (VALUE)env) (vm.c:1213).
2. Only then repoints B's ep[VM_ENV_DATA_INDEX_SPECVAL] to A's heap EP (vm.c:1142).

Between 1 and 2, B's prev EP still points to A's old stack EP, whose ep[0] is now an object pointer instead of flags. If rb_profile_frames() runs at that point, rb_vm_frame_method_entry_unchecked() (called from thread_profile_frames(), vm_backtrace.c:2143) walks from B:

• B's ME_CREF is not a method entry, so it moves on to A's stale stack EP.
• VM_ENV_LOCAL_P_UNCHECKED() reads the env pointer as flags. The pointer is aligned, so VM_ENV_FLAG_LOCAL (0x2) is clear and A is treated as non-local.
• If A's ME_CREF is not a method entry either (e.g. A is a class body or the top-level frame, which have a cref), it follows VM_ENV_PREV_EP_UNCHECKED(), i.e. ep[-1]. For a local frame with no block that is VM_BLOCK_HANDLER_NONE, so the next EP is NULL.
• It dereferences NULL → SEGV.

On 3.4, rb_yjit_invalidate_ep_is_bp() runs inside vm_make_env_each(A) after step 1, which widens the window. That explains the YJIT frame in the production backtrace. On master that call happens before the env is created, so the window is only the function return, but it is still easy to hit.

Possible fix directions

• Keep A's stack EP walkable until every child EP that points to it has been updated. For example, update B's SPECVAL before A's stack ep[0] is overwritten, or keep the env reference that the stack slot needs for GC marking somewhere other than the flags slot.
• Or make rb_vm_frame_method_entry_unchecked() stop walking when the prev EP is NULL. That avoids this crash, but on stale data it could still return a wrong method entry.

Affected versions

I could verify it locally and have it crash within 1 second with ruby rb_profile_frames_signal_crash.rb on:

• master
• 4.0
• 3.4
• 3.3
• 3.2
• Likely older versions

[Eregon (Benoit Daloze)]

• Description updated (diff)

[Eregon (Benoit Daloze)]

• Description updated (diff)

[Eregon (Benoit Daloze)]

• Assignee set to Eregon (Benoit Daloze)

[Eregon (Benoit Daloze)]

I'm working on a fix, will post the PR when it's ready here, as well as create backport PRs.

[alanwu (Alan Wu)]

CC @jhawthorn (John Hawthorn)

This sounds very similar to the issue from https://github.com/ruby/ruby/pull/16615

[Eregon (Benoit Daloze)]

PR to fix this: https://github.com/ruby/ruby/pull/19559
I'll create backport PRs after this one is merged.