Bug #20716: Different instance_method behavior in Ruby 2.7 and Ruby 3.x

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: 3.3.1
• Backport: 3.1: REQUIRED, 3.2: DONE, 3.3: DONE

[natton (Tien Truong)]

Hi, I am working on upgrading our project from 2.7 to 3.x and found a breaking change in instance_method behavior.
Here is the code snippet

module A
  def test(*args)
    super
  end
end

module B
  def test(a)
    puts a
  end
end

B.prepend(A)

a = lambda do
  puts 'lambda'
end

class C
  include B
end

B.instance_method(:test).bind(C.new).call(1)
# Ruby 2.7: 1
# Ruby 3.0: 1
# Ruby 3.1: 1

B.module_exec do
  define_method(:test, a)
end

B.instance_method(:test).bind(C.new).call
# Ruby 2.7: lambda
# Ruby 3.0: wrong number of arguments (given 0, expected 1) (ArgumentError)
# Ruby 3.1: wrong number of arguments (given 0, expected 1) (ArgumentError)

I don't know if this is a bug or a correct intention.

My specs:

• Ruby manager: rbenv
• Tested on
  • ruby 2.7.4p191 (2021-07-07 revision a21a3b7d23) [x86_64-linux]
  • ruby 3.0.7p220 (2024-04-23 revision 724a071175) [x86_64-linux]
  • ruby 3.1.4p223 (2023-03-30 revision 957bb7cb81) [x86_64-linux]
  • ruby 3.2.3 (2024-01-18 revision 52bb2ac0a6) [x86_64-linux]
  • ruby 3.3.1 (2024-04-23 revision c56cd86388) [x86_64-linux]

[jeremyevans0 (Jeremy Evans)]

This appears to be a method caching bug in super. If you remove the .call(1), then it works correctly. super_method returns the correct method. The define_method(:test, a) call should result in the method cache being cleared or ignored, but it doesn't. However, it isn't specific to define_method, regular def results in the same behavior.

Maybe related? b9007b6c548f91e88fd3f2ffa23de740431fa969

[natton (Tien Truong)]

Yes, you are correct. This isn't specific to define_method. I modified the snippet a little bit

module A
  def test(*args)
    super
  end
end

module B
  def test(a)
    puts a
  end
end

B.prepend(A)

class C
  include B
end

C.new.test(1)
# Ruby 2.7: 1
# Ruby 3.0: 1
# Ruby 3.1: 1

module B
  def test
    puts 'new test'
  end
end

C.new.test
# Ruby 2.7: 'new test'
# Ruby 3.0: wrong number of arguments (given 0, expected 1) (ArgumentError)
# Ruby 3.1: wrong number of arguments (given 0, expected 1) (ArgumentError)

[jeremyevans0 (Jeremy Evans)]

This behavior isn't related to instance_method. It occurs in the following code:

module A
  def test(*args)
    super
  end
end

module B
  def test(a)
    puts a
  end
end

class C; end

B.prepend(A)
C.include(B)

C.new.test 1

module B
  def test
    puts 'lambda'
  end
end

C.new.test

It also happens when you replace C.include(B) with C.prepend(B).

It does not happen when you replace B.prepend(A); C.include(B) with A.include(B); C.include(A). It also does not happen if a class directly prepends a module, such as in this example:

module A
  def test(*args)
    super
  end
end

class B
  def test(a)
    puts a
  end
end

B.prepend(A)

B.new.test 1

class B
  def test
    puts 'lambda'
  end
end

B.new.test

It seems like the conditions are:

• Class includes or prepends module X that prepends module Y
• Both X and Y define method M
• Y's definition of M calls super
• X's definition of M is overridden after super method call is cached

[natton (Tien Truong)]

Hi Evans,

Thank you for your confirmation.
I am new here so is there anything I can do to escalate this or how can I help to resolve this issue?

[jeremyevans0 (Jeremy Evans)]

natton (Tien Truong) wrote in #note-4:

I am new here so is there anything I can do to escalate this or how can I help to resolve this issue?

There isn't an escalation process in place in Ruby, as it is an open source project managed by volunteers. In terms of helping to resolve the issue, first, we need to determine the cause of the bug. Bisecting changes between 2.7 and 3.0 would help here to confirm the actual commit that caused the issue (the commit I listed in an early comment is just a guess). I tried bisecting myself, but I think the current version of bison doesn't work with the parse.y of 2.7, so bisecting is more challenging than usual.

After the problem has been determined, then an appropriate fix needs to be determined. This is likely more challenging, and requires knowledge and/or study of how method caching works in Ruby.

If this isn't work you are comfortable doing yourself, and it's important enough that you are willing to pay to have it fixed, you could pay a developer to do so on your behalf, or you could offer a bounty for a developer to fix it for you. There are no guarantees of success, of course, but it likely increases the odds.

Considering this bug has been around for about 4 years, and this is the first time it has been reported, this doesn't seem like a major issue, and it should be fairly easy to work around by overriding the method before calling super. That said, I would still like to see it fixed, and likely will work on it more myself at some point. However, I don't have extensive knowledge of Ruby's method caching, and there are other bugs I'm working on fixing.

[jeremyevans0 (Jeremy Evans)]

• Backport changed from 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN to 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED

I was able to track down the cause and submitted a pull request to fix this: https://github.com/ruby/ruby/pull/11582

It appears to be introduced by ad729a1d11c6c57efd2e92803b4e937db0f75252, which I wrote and which was a fix for #16736.

[natton (Tien Truong)]

Hi Evans,

Thank you for your guidance and the quick fix.
I truly appreciate it

• Status changed from Open to Closed

Applied in changeset git|6118e8a47394409b53164b60e79fadf348b97db3.

---

Fix method caching bug when including/prepend module A that prepends module B

Fix by always adding the generated iclass to the subclasses list,
otherwise the method cache for the iclass is not cleared when
the method in the module is overwritten.

Fixes [Bug #20716]

[natton (Tien Truong)]

Hi Evans,

I am sorry for my late response.
Is there a way to backport this fix to Ruby version 3.1 and 3.2

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED to 3.1: REQUIRED, 3.2: DONE, 3.3: REQUIRED

ruby_3_2 cb75fcef6feb126a67874109ce1503976a6f90cc merged revision(s) 6118e8a47394409b53164b60e79fadf348b97db3, dc64448202299633a235f310b8bf2192263f274f.

[jeremyevans0 (Jeremy Evans)]

natton (Tien Truong) wrote in #note-9:

Is there a way to backport this fix to Ruby version 3.1 and 3.2

Ruby 3.2 was recently fixed, and it will be in the next Ruby 3.2 release. Ruby 3.1 is in security fix only mode, and this is not a security fix, so I do not think it will be patched.

[natton (Tien Truong)]

Yes, thank you for the fix.

[k0kubun (Takashi Kokubun)]

• Backport changed from 3.1: REQUIRED, 3.2: DONE, 3.3: REQUIRED to 3.1: REQUIRED, 3.2: DONE, 3.3: DONE

ruby_3_3 edeb0319f7a95dfe3f9b895bcf32371dd8514726 merged revision(s) 6118e8a47394409b53164b60e79fadf348b97db3.