Bug #22178
closed
Regexp#== regression: regexps built via the `rb_reg_new` C API are no longer equal to an identical literal
Description
On current master, two Regexp objects that have the same options, the same regexp encoding (Regexp#encoding),
and byte-identical sources are no longer == if their source strings have different encodings (e.g. ASCII-8BIT vs US-ASCII).
This worked on 4.0 and earlier.
Such a regexp is produced by the public C API rb_reg_new(), which does not normalize the source encoding,
so an ASCII-only pattern keeps an ASCII-8BIT source instead of being promoted to US-ASCII.
Reproduction¶
require "fiddle"
libruby = Fiddle.dlopen(nil)
RB_REG_NEW = Fiddle::Function.new(
libruby["rb_reg_new"],
[Fiddle::TYPE_VOIDP, Fiddle::TYPE_LONG, Fiddle::TYPE_INT],
Fiddle::TYPE_VOIDP
)
def reg_via_c_api(src, options = 0) # what a C extension using
ptr = RB_REG_NEW.call(Fiddle::Pointer[src], src.bytesize, options)
Fiddle.dlunwrap(ptr.to_i) # rb_reg_new() produces
end
c = reg_via_c_api("^[0-9]") # rb_reg_new() -> source ASCII-8BIT
lit = /^[0-9]/ # literal -> source US-ASCII
p c.source.encoding # ASCII-8BIT
p lit.source.encoding # US-ASCII
p c.encoding # US-ASCII
p lit.encoding # US-ASCII
p(c.options == lit.options) # true
p(c.source == lit.source) # true (byte-identical)
p(c == lit) # 4.0: true / master: false
Output on master:
#<Encoding:ASCII-8BIT>
#<Encoding:US-ASCII>
#<Encoding:US-ASCII>
#<Encoding:US-ASCII>
true
true
false # <-- was true on 4.0.5
Cause¶
Bisects to commit d663de3fc43a8bb54bbdde12764b54bff4369da0 ("re.c: Simplify rb_reg_equal", 2026-06-24).
Confirmed by building the two adjacent commits.
| commit | c == lit |
|---|---|
bb75c2893a (parent) |
true |
d663de3fc4 |
false |
rb_reg_equal changed from comparing regexp encoding + source content to comparing the source fstring identity:
// before
if (RREGEXP_SRC_LEN(re1) != RREGEXP_SRC_LEN(re2)) return Qfalse;
if (ENCODING_GET(re1) != ENCODING_GET(re2)) return Qfalse;
return RBOOL(memcmp(RREGEXP_SRC_PTR(re1), RREGEXP_SRC_PTR(re2), RREGEXP_SRC_LEN(re1)) == 0);
// after
if (RREGEXP_SRC(re1) != RREGEXP_SRC(re2)) return Qfalse; // fstring pointer compare
return RBOOL(ENCODING_GET(re1) == ENCODING_GET(re2));
Because fstring interning is encoding-sensitive, a US-ASCII "^[0-9]" and an ASCII-8BIT "^[0-9]" are distinct fstrings, so the new pointer comparison returns false where the old content+encoding comparison returned true.
Question¶
Is this behavior change intentional, or a side effect of the optimization?
The commit message describes d663de3fc4 as a pure optimization and doesn't mention making the source string's encoding significant to Regexp#==, so it reads like an unintended side effect: the optimization assumes every regexp source is a normalized fstring, but rb_reg_new() can produce one whose source is a valid fstring with a non-normalized (ASCII-8BIT) encoding.
Updated by