Bug #19739
closedString coderange not cleared by String#slice!
Description
Hello,
Key cannot be found in a Hash when slice! method is applied to the key. I cannot reproduce this behaviour with sub! or gsub! methods:
hash = {}
key = "ABC OÜ"
key.slice!(/ oü$/i)
# key.sub!(/ oü$/i, '')
# key.gsub!(/ oü$/i, '')
hash[key] = true
"#{key} in #{hash}: value is #{hash[key]}"
=> "ABC in {\"ABC\"=>true}: value is "
Also it seems that ruby 2.7.6 and 3.2.2 don't have this issue.
Updated by byroot (Jean Boussier) about 1 year ago
That is very interesting.
The two strings are identical (aside from one being frozen):
require "objspace"
hash = {}
key = "ABC OÜ"
p key.hash
key.slice!(/ oü$/i)
p key.hash
puts ObjectSpace.dump(key)
hash[key] = true
p "#{key} in #{hash}: value is #{hash[key].inspect}"
puts ObjectSpace.dump(hash.keys.first)
p [hash.keys.first, key, hash.keys.first == key]
p [hash.keys.first.hash, key.hash]
1883393806954673666
-2878200511277543492
{"address":"0x10129f048", "type":"STRING", "class":"0x1011c2418", "embedded":true, "bytesize":3, "encoding":"UTF-8", "memsize":40, "flags":{"wb_protected":true}}
"ABC in {\"ABC\"=>true}: value is nil"
{"address":"0x10129ebc0", "type":"STRING", "class":"0x1011c2418", "frozen":true, "embedded":true, "fstring":true, "bytesize":3, "value":"ABC", "encoding":"UTF-8", "memsize":40, "flags":{"wb_protected":true}}
["ABC", "ABC", true]
[-2878200511277543491, -2878200511277543491]
The hash code is the same, and ==
returns true.
I had a quick look at recent changes in String#slice!
that could explain why it's fixed in 3.2, but didn't spot anything yet.
What is interesting though is that the first access to key.hash
returns -2878200511277543492
, but then later it consistently returns -2878200511277543491
. I wonder if there is an off by one bug somewhere.
Updated by byroot (Jean Boussier) about 1 year ago
- Backport changed from 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN to 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: DONTNEED
I suspect https://github.com/ruby/ruby/pull/5867 is what fixed it, but don't have time to confirm just yet.
Updated by byroot (Jean Boussier) about 1 year ago
- Backport changed from 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: DONTNEED to 3.0: REQUIRED, 3.1: REQUIRED, 3.2: DONTNEED
Ok, so after bisecting it appears that what fixed it was: https://github.com/ruby/ruby/commit/b0b9f7201acab05c2a3ad92c3043a1f01df3e17f
This won't be easy to backport though, as it was a performance patch. Especially since it had to be fixed later with https://github.com/ruby/ruby/pull/7437/commits/2d0c804e5640475202b7c24559bbe1b151367ebf
We might need to craft a patch specifically to fix that bug.
Updated by byroot (Jean Boussier) about 1 year ago
OK fix is here: https://github.com/ruby/ruby/pull/7965
Updated by byroot (Jean Boussier) about 1 year ago
cc @nagachika (Tomoyuki Chikanaga) is a PR targeting ruby_3_1
OK? Or should I submit the patch differently?
Updated by byroot (Jean Boussier) about 1 year ago
- Subject changed from Key cannot be found in a Hash when slice! method is applied to the key to String coderange not cleared by String#slice!
Updated by hsbt (Hiroshi SHIBATA) about 1 year ago
@byroot (Jean Boussier) ruby_3_1
is handled by @usa (Usaku NAKAMURA) now.
Updated by hsbt (Hiroshi SHIBATA) about 1 year ago
- Status changed from Open to Closed
Updated by byroot (Jean Boussier) about 1 year ago
The commit to backport would be: https://github.com/Shopify/ruby/commit/3b351ee62d4206bb72301c2e98dcb173f1e35be7
Updated by usa (Usaku NAKAMURA) about 1 year ago
- Backport changed from 3.0: REQUIRED, 3.1: REQUIRED, 3.2: DONTNEED to 3.0: REQUIRED, 3.1: DONE, 3.2: DONTNEED
ruby_3_1 7e926c4d8c2e785d174161b38a8a688b19ea4b99.