Bug #15543
closedrb_str_set_len should clear code range
Description
Calling rb_str_set_len
on a String
could alter the code range. I think this hasn't been much of an issue because of pure luck rather than anything that was deliberately designed. If called on a string that already has a CR_UNKNOWN
code range, there's no problem because the correct values will lazily calculated.
An example invocation that could be problematic, using helper methods for writing C API specs from the Ruby Spec Suite, looks like:
@str = "abcdefghij"[0..-1]
@s.rb_str_set_len(@str, 1)
@str.should == "a"
@str.force_encoding(Encoding::UTF_8)
@str.valid_encoding?.should == true
@s.RSTRING_PTR_set(@str, 1, 'B'.ord)
@s.RSTRING_PTR_set(@str, 2, 0x80)
@s.rb_str_set_len(@str, 3)
p @str
@str.valid_encoding?.should == false # This line fails because the cached code range hasn't been updated.
The first call to valid_encoding?
forces the code range to be calculated for the string and it's determined to be CR_7BIT
. Then rb_str_set_len
is called, simulating the splitting the bytes of a valid UTF-8 multi-byte character. Here, the code range is still cached as CR_7BIT
, but the byte sequence is invalid for UTF-8.
I think the fix is a simple matter of clearing the code range in rb_str_set_len
, but I'd appreciate a review of my analysis.
Updated by nobu (Nobuyoshi Nakada) almost 6 years ago
- Status changed from Open to Rejected
It isn't an issue of rb_str_set_len
.
The string isn't valid encoding by @s.RSTRING_PTR_set(@str, 2, 0x80)
.
You should call rb_str_modify
before modifying a string object.
Updated by Eregon (Benoit Daloze) almost 6 years ago
I think few people know about this, and rb_str_modify
isn't documented in doc/extension.rdoc.
I guess some C extensions ignore this requirement and could have bugs because of it.
Isn't it simpler and safer to clear the coderange, as it's done for, e.g., rb_str_resize()
?
Updated by nobu (Nobuyoshi Nakada) almost 6 years ago
rb_str_set_len
is not the true problem.
Should RSTRING_PTR
make the object unshared and clear the code range?
Or enclose it only for the core and prohibit in extension libraries?
Hmmm, the last choice may be the best, I think.
Updated by Eregon (Benoit Daloze) almost 6 years ago
nobu (Nobuyoshi Nakada) wrote:
rb_str_set_len
is not the true problem.
ShouldRSTRING_PTR
make the object unshared and clear the code range?
That sounds safer, because indeed as soon as the C code can access the char*
it can change it.
Or enclose it only for the core and prohibit in extension libraries?
I'm not sure what you mean?
Do you mean removing rb_str_set_len() or RSTRING_PTR from the public C-API?
Or do you mean in core, a special version of RSTRING_PTR would be used?
Or something else?