Project

General

Profile

Actions

Bug #15543

closed

rb_str_set_len should clear code range

Added by nirvdrum (Kevin Menard) over 2 years ago. Updated over 2 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Target version:
-
ruby -v:
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-linux]
[ruby-core:91129]

Description

Calling rb_str_set_len on a String could alter the code range. I think this hasn't been much of an issue because of pure luck rather than anything that was deliberately designed. If called on a string that already has a CR_UNKNOWN code range, there's no problem because the correct values will lazily calculated.

An example invocation that could be problematic, using helper methods for writing C API specs from the Ruby Spec Suite, looks like:

@str = "abcdefghij"[0..-1]
@s.rb_str_set_len(@str, 1)
@str.should == "a"

@str.force_encoding(Encoding::UTF_8)
@str.valid_encoding?.should == true
@s.RSTRING_PTR_set(@str, 1, 'B'.ord)
@s.RSTRING_PTR_set(@str, 2, 0x80)
@s.rb_str_set_len(@str, 3)

p @str
@str.valid_encoding?.should == false # This line fails because the cached code range hasn't been updated.

The first call to valid_encoding? forces the code range to be calculated for the string and it's determined to be CR_7BIT. Then rb_str_set_len is called, simulating the splitting the bytes of a valid UTF-8 multi-byte character. Here, the code range is still cached as CR_7BIT, but the byte sequence is invalid for UTF-8.

I think the fix is a simple matter of clearing the code range in rb_str_set_len, but I'd appreciate a review of my analysis.

Actions

Also available in: Atom PDF