Bug #20871: Including methods in Enumerable doesn't make them available in Array

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: 3.3.6
• Backport: 3.1: REQUIRED, 3.2: DONE, 3.3: REQUIRED

[sanderd17 (Sander Deryckere)]

Today, our CI pipeline started failing after the automatic update from v3.3.5 to v3.3.6.

After researching, it turned out that our core extensions to the Array class weren't loaded anymore.

The core-extensions code looks like this:

module CoreExt
  module Enumerable
    def average
      sum(&:to_f) / count if any?
    end

    # def ...
  end
end

Enumerable.include CoreExt::Enumerable

After some debugging, it turned out that the average method was included in the instance_methods of Enumerable, but not in the instance_methods of Array.

Explicitly adding Array.include CoreExt::Enumerable fixes CI for our case.

The very strange thing is that it only happens on a release branch we are still maintaining. It doesn't happen on our main development branch (which also updated to v3.3.6 today). I have been unable to find the difference between both branches so far (they diverged a couple of months ago, but the base system is still regularly updated on both, and pretty similar).

After some digging around, I assume this commit is related to our issue: https://github.com/ruby/ruby/commit/edeb0319f7a95dfe3f9b895bcf32371dd8514726

[sanderd17 (Sander Deryckere)]

Forgot to mention.

We use the parallel_tests gem to speed up our specs in CI.

On my local environment, with synchronous tests, I haven't been able to reproduce the same behaviour either.

[jeremyevans0 (Jeremy Evans)]

• Status changed from Open to Feedback

The example works as expected on Ruby 3.3.6. It seems unlikely the commit you are referencing is related, because the commits fixes an issue where the method cache is not updated on a change, it wouldn't cause the method not to show up at all. Are you able to provide a self-contained reproducible example for this issue?

[rolf (Rolf T)]

We are running into the exact same problem. I was not able to reproduce it except as part of a Rails application. Maybe that's not exactly helpful, but hopefully this helps zoom in further on the issue. The problem seems to be introduced in Ruby 3.3.6.

A working reproduction is here: https://github.com/rolftimmermans/ruby-20871

module Ext
  def foo
    "foo"
  end
end

Enumerable.include(Ext)

With Ruby 3.3.6 via irb there seems to be no issue:

% irb                    
irb(main):001> RUBY_VERSION
=> "3.3.6"
irb(main):002> require_relative "lib/ext"
=> true
irb(main):003> [].foo
=> "foo"

With Ruby 3.3.6 via Rails console the bug manifests itself:

% rails c
Loading development environment (Rails 7.1.5)
irb(main):001> RUBY_VERSION
=> "3.3.6"
irb(main):002> require_relative "lib/ext"
=> true
irb(main):003> [].foo
(irb):3:in `<main>': undefined method `foo' for an instance of Array (NoMethodError)

[].foo
  ^^^^

With Ruby 3.3.5 via Rails console everything is working fine:

% rails c     
/Users/rolftimmermans/.rubies/ruby-3.3.5/lib/ruby/3.3.0/json/generic_object.rb:2: warning: /Users/rolftimmermans/.rubies/ruby-3.3.5/lib/ruby/3.3.0/ostruct.rb was loaded from the standard library, but will no longer be part of the default gems starting from Ruby 3.5.0.
You can add ostruct to your Gemfile or gemspec to silence this warning.
Loading development environment (Rails 7.1.5)
irb(main):001> RUBY_VERSION
=> "3.3.5"
irb(main):002> require_relative "lib/ext"
=> true
irb(main):003> [].foo
=> "foo"

[rolf (Rolf T)]

I managed to narrow it down a bit further. With the versions specified in https://github.com/rolftimmermans/ruby-20871/blob/main/Gemfile.lock:

% irb
irb(main):001> require "active_support"
irb(main):002> require "active_support/json/decoding"
irb(main):003> 
irb(main):004* class Foo
irb(main):005*   include Enumerable
irb(main):006> end
irb(main):007> 
irb(main):008* module Ext
irb(main):009*   def foo
irb(main):010*     "foo"
irb(main):011*   end
irb(main):012> end
irb(main):013> 
irb(main):014> Enumerable.include(Ext)
irb(main):015> 
=> Enumerable
irb(main):016> [].foo
(irb):16:in `<main>': undefined method `foo' for an instance of Array (NoMethodError)

[].foo
  ^^^^
	from <internal:kernel>:187:in `loop'
	from /Users/rolftimmermans/.gem/ruby/3.3.6/gems/irb-1.14.1/exe/irb:9:in `<top (required)>'
	from /Users/rolftimmermans/.rubies/ruby-3.3.6/bin/irb:25:in `load'
	from /Users/rolftimmermans/.rubies/ruby-3.3.6/bin/irb:25:in `<main>'

[tdeo (Thierry Deo)]

I'm running into a similar situation which I managed to reproduce with a plain ruby console and no external dependencies:

$ irb
irb(main):001> 
module Bar; def bar; 'bar'; end; end
module Foo; def foo; 'foo'; end; end

Enumerable.prepend Bar
Class.new { include Enumerable }

Enumerable.include Foo

puts [].foo

(irb):9:in `<main>': undefined method `foo' for an instance of Array (NoMethodError)

puts [].foo
       ^^^^
        from <internal:kernel>:187:in `loop'
        from /Users/thierry/.asdf/installs/ruby/3.3.6/lib/ruby/gems/3.3.0/gems/irb-1.14.1/exe/irb:9:in `<top (required)>'
        from /Users/thierry/.asdf/installs/ruby/3.3.6/bin/irb:25:in `load'
        from /Users/thierry/.asdf/installs/ruby/3.3.6/bin/irb:25:in `<main>'

Removing either of the Enumerable.prepend or Class.new lines makes the bug go away

[sanderd17 (Sander Deryckere)]

Thanks @rolf (Rolf T) and @tdeo (Thierry Deo) for finding minimal examples!

Why is this ticket marked as closed now?

[jeremyevans0 (Jeremy Evans)]

• Status changed from Feedback to Open

sanderd17 (Sander Deryckere) wrote in #note-6:

Thanks @rolf (Rolf T) and @tdeo (Thierry Deo) for finding minimal examples!

Why is this ticket marked as closed now?

It was in Feedback status, which shows the Closed button. Now that we've received feedback, including a self contained example, I've moved it back to Open.

[jeremyevans0 (Jeremy Evans)]

I was able to reproduce this but in Ruby 3.2.6, 3.3.6, and master. Backing out 6118e8a47394409b53164b60e79fadf348b97db3 (the commit to master that edeb0319f7a95dfe3f9b895bcf32371dd8514726 was based on), fixes the problem, so I was too quick to dismiss that as a cause. My apologies for that.

We could revert 6118e8a47394409b53164b60e79fadf348b97db3 in master (with similar reverts for 3.2 and 3.3), and reopen #20716.

Looking at the change in 6118e8a47394409b53164b60e79fadf348b97db3, it seems like adding module Foo to the subclasses list is breaking something.

I'm not sure when I'll have time to try to fix this, it might not be until next month. So if anyone else has time and can find a way to fix #20716 without causing the breakage in this issue, that would be awesome.

[jeremyevans0 (Jeremy Evans)]

• Backport changed from 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN to 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED

I had some extra time today, and I was able to find the cause, and once I found the cause, the fix was trivial: https://github.com/ruby/ruby/pull/12125

The root cause is a bug in the initial commit to allow Module#include to affect classes and modules that that have already included the receiver, 3556a834a2847e52162d1d3302d4c64390df1694 (which I wrote).

• Status changed from Open to Closed

Applied in changeset git|3b7892b6e4d1a1a5d6019987f9b46ed443dd104f.

---

Fix a bug in rb_include_module that stops nested inclusion into module subclasses

This bug was present since the code was originally added by me
in 3556a834a2847e52162d1d3302d4c64390df1694.

Fixes [Bug #20871]

[sanderd17 (Sander Deryckere)]

So the bug has always been present, but was hidden by a different caching issue.

Great find, wouldn't have known where to start.

Thank you so much for taking this up so quickly!

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED to 3.1: REQUIRED, 3.2: DONE, 3.3: REQUIRED

ruby_3_2 bb065f08441aab8f97b45bd3f1600202547c9532 merged revision(s) 3b7892b6e4d1a1a5d6019987f9b46ed443dd104f.