Bug #20653: Memory leak in String#start_with? when regexp times out

• Status: Open
• Assignee: -
• Target version: -
• Backport: 3.1: UNKNOWN, 3.2: WONTFIX, 3.3: REQUIRED

[peterzhu2118 (Peter Zhu)]

GitHub PR: https://github.com/ruby/ruby/pull/11247

This commit refactors how Onigmo handles timeout. Instead of raising a timeout error, onig_search will return a ONIGERR_TIMEOUT which the caller can free memory, and then raise a timeout error.

This fixes a memory leak in String#start_with when the regexp times out. For example:

regex = Regexp.new("^#{"(a*)" * 10_000}x$", timeout: 0.000001)
str = "a" * 1000000 + "x"

10.times do
  100.times do
    str.start_with?(regex)
  rescue
  end

  puts `ps -o rss= -p #{$$}`
end

Before:

33216
51936
71152
81728
97152
103248
120384
133392
133520
133616

After:

14912
15376
15824
15824
16128
16128
16144
16144
16160
16160