Bug #22174: Set operations (&, ^, collect!, flatten, classify, divide) do not preserve compare_by_identity

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 4.0.5 (2026-05-20 revision 64336ffd0e) +PRISM [arm64-darwin25]
• Backport: 3.3: DONTNEED, 3.4: DONTNEED, 4.0: REQUIRED

[gil.desmarais (Gil Desmarais)]

Since Set was transitioned to a C-level core class in Ruby 4.0, several operations that allocate new sets or partition subsets (&, ^, collect!, flatten, classify, and divide) lost the compare_by_identity behavior for the receiver set or its subsets.

• For &, collect!, flatten, classify, and divide, the resulting sets/subsets are allocated without propagating the compare_by_identity flag (returning false for #compare_by_identity?).
• For ^ (XOR) with a generic Enumerable, the returned set has the flag set (via dup), but it allocates a temporary set tmp to hold the RHS elements without propagating the compare_by_identity flag. This causes the RHS elements to be deduplicated incorrectly using value equality instead of identity equality.

A pull request has been opened with the fix and tests: https://github.com/ruby/ruby/pull/17633

Reproduction

require 'set'

# 1. Intersection &
s1 = Set.new.compare_by_identity
s2 = Set.new
s1 << "a"
s2 << "a"
puts "Intersection preserves compare_by_identity: #{(s1 & s2).compare_by_identity?}"
# Expected: true
# Actual:   false

# 2. XOR ^ (with duplicate objects by value on RHS)
s_xor = Set.new.compare_by_identity
x1 = +"x"
x2 = +"x"
result = s_xor ^ [x1, x2]
puts "XOR with Enumerable preserves identity comparison: #{result.size == 2}"
# Expected: true (size should be 2, because x1 and x2 are distinct objects)
# Actual:   false (size is 1)

# 3. collect! / map!
s_collect = Set.new(["a", "b"]).compare_by_identity
s_collect.collect! { |x| x }
puts "collect! preserves compare_by_identity: #{s_collect.compare_by_identity?}"
# Expected: true
# Actual:   false

# 4. flatten
s_flat = Set.new([Set.new([1])]).compare_by_identity
puts "flatten preserves compare_by_identity: #{s_flat.flatten.compare_by_identity?}"
# Expected: true
# Actual:   false

# 5. classify
s_classify = Set.new(["a", "b"]).compare_by_identity
classified = s_classify.classify { |x| x }
puts "classify subsets preserve compare_by_identity: #{classified.values.all?(&:compare_by_identity?)}"
# Expected: true
# Actual:   false

# 6. divide
s_divide = Set.new(["a", "b"]).compare_by_identity
divided = s_divide.divide { |x| x }
puts "divide subsets preserve compare_by_identity: #{divided.all?(&:compare_by_identity?)}"
# Expected: true
# Actual:   false

[jeremyevans0 (Jeremy Evans)]

• Backport changed from 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN to 3.3: DONTNEED, 3.4: DONTNEED, 4.0: REQUIRED

[jeremyevans0 (Jeremy Evans)]

• Backport changed from 3.3: DONTNEED, 3.4: DONTNEED, 4.0: REQUIRED to 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN

From my testing with the example given, Set has returned false for all of these operations since Set#compare_by_identity was added in Ruby 2.4:

$ ruby24 -v t.rb
ruby 2.4.9p362 (2019-10-02 revision 67824) [x86_64-openbsd]
Intersection preserves compare_by_identity: false
XOR with Enumerable preserves identity comparison: false
collect! preserves compare_by_identity: false
flatten preserves compare_by_identity: false
classify subsets preserve compare_by_identity: false
divide subsets preserve compare_by_identity: false

$ ruby34 -v t.rb
ruby 3.4.10 (2026-06-30 revision 2b0b7728dc) +PRISM [x86_64-openbsd]
Intersection preserves compare_by_identity: false
XOR with Enumerable preserves identity comparison: false
collect! preserves compare_by_identity: false
flatten preserves compare_by_identity: false
classify subsets preserve compare_by_identity: false
divide subsets preserve compare_by_identity: false

$ ruby40 -v t.rb
ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [x86_64-openbsd]
Intersection preserves compare_by_identity: false
XOR with Enumerable preserves identity comparison: false
collect! preserves compare_by_identity: false
flatten preserves compare_by_identity: false
classify subsets preserve compare_by_identity: false
divide subsets preserve compare_by_identity: false

So the core Set implementation returning false for these operations is backwards compatible. It would be backwards incompatible to change them to return true. Maybe we should change them, but we should realize that doing so is breaking backwards compatibility and not restoring backwards compatibility.

@knu (Akinori MUSHA) What are your thoughts on this? Should we break backwards compatibility here? If so, should we backport it to both Ruby 4.0 (core C implementation) and/or Ruby 3.4 (stdlib Ruby implementation)? ruby/set has been archived, so I assume we would need to unarchive it to fix the issue in Ruby 3.4.

Personally, I think we should handle this on a case by case basis:

• collect!/map!: Unsetting the compare_by_identity flag does not make sense, so I think we should change these.
• divide/classify: It seems reasonable to keep compare_by_identity in the returned sets, but I'm not sure it's worth the backwards compatibility breakage.
• &/^: If both receiver and argument have compare_by_identity, it seems best that the resulting set uses compare_by_identity. However, if the receiver has compare_by_identity and the argument does not (or vice versa), I'm not sure we should necessarily use the receiver's setting. I think the behavior here should be that b & a and a & b return a consistent value in regards to compare_by_identity if a has compare_by_identity and b does not.
• flatten: This is a transformation of some kind, and Ruby doesn't necessarily keep compare_by_identity across transformations. For example, Hash#transform_values keeps compare_by_identity, but Hash#transform_keys does not, and sets and hash keys are closely related (set is basically a hash with keys and no values). Not sure it's worth the backwards compatibility breakage to change this.

[knu (Akinori MUSHA)]

I'd leave Ruby 3.4 and the pure Ruby implementation just as they are.

As for Ruby 4.x I'd expect:

• collect!/map! retain the identity flag of the receiver.
• divide/classify generate subsets inheriting the flag of the receiver.
• &/^ return a set inheriting the flag of the LHS set.
• flatten returns a set inheriting the flag of the receiver.

And on these points, I would prefer consistency with what users who explicitly choose to use the flag would expect from this feature over backward compatibility.

[jeremyevans0 (Jeremy Evans)]

OK. I submitted https://github.com/ruby/ruby/pull/18848 to fix this in master (same as https://github.com/ruby/ruby/pull/17633 but squashed and with fixed specs). It leaves Ruby 4.0 behavior unspecified. After it is merged, I'll submit a backport PR for Ruby 4.0. After that is merged, I'll update the specs to specify the Ruby 4.0 behavior matches the Ruby 4.1 behavior.

[jeremyevans0 (Jeremy Evans)]

• Status changed from Open to Closed
• Backport changed from 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN to 3.3: DONTNEED, 3.4: DONTNEED, 4.0: REQUIRED

Fixed in f069efc3f69a89188bab4ac70c690aecce59023b

[jeremyevans0 (Jeremy Evans)]

4.0 backport PR: https://github.com/ruby/ruby/pull/18992