Bug #18631
closedRange check breaks multiplex backreferences in regular expressions
Description
The regular expression engine can sometimes produce wrong results when using multiplex backreferences near the end of the input string.
irb(main):001:0> /(?<x>a)(?<x>aa)\k<x>/.match("aaaaa")
=> #<MatchData "aaaaa" x:"a" x:"aa">
irb(main):002:0> /(?<x>a)(?<x>aa)\k<x>/.match("aaaa")
=> nil
irb(main):003:0> /(?<x>a)(?<x>aa)\k<x>/.match("aaaab")
=> #<MatchData "aaaa" x:"a" x:"aa">
The second and third calls to match
should produce the same result.
The cause is the DATA_ENSURE(n)
macro in the OP_BACKREF_MULTI
case in regexec.c
(https://github.com/ruby/ruby/blob/master/regexec.c#L2646). Instead of using continue
to try to match the other referents for the backref (as all the other branches do), the DATA_ENSURE
macro uses goto fail
and so skips the other referents of the multiplex backref. This means that after failing the range check, no other referent can match. By extending the input string in the third example above, we have avoided the bug and got the correct result.
Updated by jeremyevans0 (Jeremy Evans) over 2 years ago
I don't know much about this code, but the approach described does fix the issue. I submitted a pull request for it: https://github.com/ruby/ruby/pull/5710
Updated by jeremyevans (Jeremy Evans) over 2 years ago
- Status changed from Open to Closed
Applied in changeset git|6d3f447aecfb56f7d3edbdf9cc68e748e150d7d8.
Fix multiplex backreferencs near end of string in regexp match
Idea from Jirka Marsik.
Fixes [Bug #18631]
Updated by nagachika (Tomoyuki Chikanaga) over 2 years ago
- Backport changed from 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN to 2.7: REQUIRED, 3.0: REQUIRED, 3.1: REQUIRED
Updated by nagachika (Tomoyuki Chikanaga) about 2 years ago
- Backport changed from 2.7: REQUIRED, 3.0: REQUIRED, 3.1: REQUIRED to 2.7: REQUIRED, 3.0: REQUIRED, 3.1: DONE
ruby_3_1 6b8bf6ba5d1893b3c933c2de3ada1eb59a94b644 merged revision(s) 6d3f447aecfb56f7d3edbdf9cc68e748e150d7d8.