Bug #22096: Freeing a mutex locked by a fiber inside fiber scheduler can crash

• Status: Closed
• Assignee: luke-gru (Luke Gruber)
• Target version: 4.1
• Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: DONE

[luke-gru (Luke Gruber)]

While freeing a locked mutex, if there is a waiter for the mutex it could call rb_fiber_scheduler_unblock which calls into Ruby code.

The function rb_mutex_unlock_th was a footgun because it was used during normal Ruby code and also during GC. The fix is to not call this function during GC and simply remove the mutex from th->keeping_mutexes.

PR

Backport PR

[luke-gru (Luke Gruber)]

• Description updated (diff)

[byroot (Jean Boussier)]

• Backport changed from 4.0: REQUIRED, 3.4: UNKNOWN, 4.0: UNKNOWN to 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: REQUIRED

[mame (Yusuke Endoh)]

Is it difficult to write a regression test for this?

[luke-gru (Luke Gruber)]

mame (Yusuke Endoh) wrote in #note-3:

Is it difficult to write a regression test for this?

I have another PR for a similar issue that has 2 regressions tests. Those tests also cover this bug, but the fix itself is more involved so I made it a separate PR. This fix is for a code issue, whereas the other fixes a design issue. When that PR lands it will also need to be backported. If it makes more sense, I can combine the 2 PRs into 1 backport PR when the other gets merged.

• Status changed from Open to Closed

Applied in changeset git|f5373f55bb8dea60508e7fd0ca2f344f2d6790e4.

---

Fix mutex_free so it doesn't call into Ruby code during GC

While freeing a locked mutex, if there is a waiter for the
mutex it could call rb_fiber_scheduler_unblock which calls
into Ruby code.

The function rb_mutex_unlock_th was a footgun because it was used
during normal Ruby code and also during GC. The fix is to not call this
function during GC and simply remove the mutex from th->keeping_mutexes.

Fixes [Bug #22096] (Backport)

[luke-gru (Luke Gruber)]

• Backport changed from 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: REQUIRED to 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: DONE