Bug #19989: Fix refinement refine performance

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) +YJIT [arm64-darwin22.4.0]
• Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN

[bkuhlmann (Brooke Kuhlmann)]

Why

Hello. 👋 Would it be possible to improve the performance of refining an object as I'm seeing a significant performance cost? I know some of this is already known but, when searching through existing issues, I couldn't find any issue that specifically mentioned or called attention to refine performance.

My C skills are not that great but it seems that rb_mod_refine (source) might be the primary source of complexity and performance degradation?

By fixing this, we could improve refinement performance and encourage more adoption since refinements are a powerful way to make small adjustments to the language without resorting to more complicated solutions -- or worse -- using monkey patches.

How

Here's the benchmark I'm using:

#! /usr/bin/env ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

Benchmark.ips do |benchmark|
  benchmark.config time: 5, warmup: 2

  benchmark.report "With" do
    Module.new do
      refine String do
        def dud = true
      end
    end
  end

  benchmark.report "Without" do
    Module.new do
      def dud = true
    end
  end

  benchmark.compare!
end

When running the above, I'm getting the following results (this is a random sample):

Ruby 2.6.6 (ruby 2.6.6p146 (2020-03-31 revision 67876) [-darwin22])

ℹ️ Temporarily refactored the benchmark to not use endless methods for this version.

Warming up --------------------------------------
                With   745.000  i/100ms
             Without    67.593k i/100ms
Calculating -------------------------------------
                With     10.849k (±168.6%) i/s -     18.625k in   5.359230s
             Without    680.199k (± 5.8%) i/s -      3.447M in   5.085558s

Comparison:
             Without:   680198.8 i/s
                With:    10849.1 i/s - 62.70x  slower

Ruby 3.2.2 (ruby 3.2.2 (2023-03-30 revision e51014f9c0) +YJIT [arm64-darwin22.4.0])

Warming up --------------------------------------
                With   959.000  i/100ms
             Without   297.091k i/100ms
Calculating -------------------------------------
                With     21.856k (±198.5%) i/s -     30.688k in   5.509103s
             Without      3.281M (± 5.8%) i/s -     16.637M in   5.082246s

Comparison:
             Without:  3281310.6 i/s
                With:    21856.1 i/s - 150.13x  slower

Ruby 3.3.3 Preview 2 (ruby 3.3.0preview2 (2023-09-14 master e50fcca9a7) +YJIT [arm64-darwin22.6.0])

Warming up --------------------------------------
                With     1.044k i/100ms
             Without   288.264k i/100ms
Calculating -------------------------------------
                With     24.044k (±192.0%) i/s -     29.232k in   5.305651s
             Without      3.096M (± 8.0%) i/s -     15.566M in   5.059470s

Comparison:
             Without:  3095640.6 i/s
                With:    24043.8 i/s - 128.75x  slower

Notes

Here's the environment I'm using:

• macOS: 13.6.1, Apple M1

[ahorek (Pavel Rosický)]

By fixing this, we could improve refinement performance and encourage more adoption since refinements are a powerful way to make small adjustments to the language without resorting to more complicated solutions -- or worse -- using monkey patches.

this shouldn't really affect the runtime performance of real programs unless you're dynamically creating refined modules like in your benchmark (which is also slow). Do you have an example where this could be a problem?

[bkuhlmann (Brooke Kuhlmann)]

this shouldn't really affect the runtime performance of real programs

Thanks and fair point. I'm using the above benchmark to track performance across Ruby versions. I agree, dynamically creating refinements isn't normal but wanted to simulate the defining of a refinement. I know there is cost in defining the module each time but am trying to zero in on the use of refine, specifically. Maybe this is the wrong way to tackle the problem and I should be using a different benchmark?

Do you have an example where this could be a problem?

I'm afraid not. Well, not as easily encapsulated, that is. I do have the following:

• Benchmarks: This is an attempt to benchmark refinement performance from multiple angles (along with other non-refinement based benchmarks).
• Refinements: The gem -- which I maintain -- and have been using in multiple projects. Also the impetus for wanting to investigate the cost of defining refinements.

[Eregon (Benoit Daloze)]

Maybe this is the wrong way to tackle the problem and I should be using a different benchmark?

Yes, the performance of refine itself should be irrelevant, it's paid once during startup (unless it would amount to taking a significant overhead to load a real app).
Just like calling Module.new in the inner loop of an app is a mistake, those tools (module definition, defining refinements) are meant to be used during code loading and very little after.

[bkuhlmann (Brooke Kuhlmann)]

OK, thanks. Make sense. Feel free to close this.

[jeremyevans0 (Jeremy Evans)]

• Status changed from Open to Closed