Bug #19989
closedFix refinement refine performance
Description
Why¶
Hello. 👋 Would it be possible to improve the performance of refining an object as I'm seeing a significant performance cost? I know some of this is already known but, when searching through existing issues, I couldn't find any issue that specifically mentioned or called attention to refine
performance.
My C skills are not that great but it seems that rb_mod_refine
(source) might be the primary source of complexity and performance degradation?
By fixing this, we could improve refinement performance and encourage more adoption since refinements are a powerful way to make small adjustments to the language without resorting to more complicated solutions -- or worse -- using monkey patches.
How¶
Here's the benchmark I'm using:
#! /usr/bin/env ruby
# frozen_string_literal: true
require "bundler/inline"
gemfile true do
source "https://rubygems.org"
gem "benchmark-ips"
end
Benchmark.ips do |benchmark|
benchmark.config time: 5, warmup: 2
benchmark.report "With" do
Module.new do
refine String do
def dud = true
end
end
end
benchmark.report "Without" do
Module.new do
def dud = true
end
end
benchmark.compare!
end
When running the above, I'm getting the following results (this is a random sample):
Ruby 2.6.6 (ruby 2.6.6p146 (2020-03-31 revision 67876) [-darwin22])
ℹ️ Temporarily refactored the benchmark to not use endless methods for this version.
Warming up --------------------------------------
With 745.000 i/100ms
Without 67.593k i/100ms
Calculating -------------------------------------
With 10.849k (±168.6%) i/s - 18.625k in 5.359230s
Without 680.199k (± 5.8%) i/s - 3.447M in 5.085558s
Comparison:
Without: 680198.8 i/s
With: 10849.1 i/s - 62.70x slower
Ruby 3.2.2 (ruby 3.2.2 (2023-03-30 revision e51014f9c0) +YJIT [arm64-darwin22.4.0])
Warming up --------------------------------------
With 959.000 i/100ms
Without 297.091k i/100ms
Calculating -------------------------------------
With 21.856k (±198.5%) i/s - 30.688k in 5.509103s
Without 3.281M (± 5.8%) i/s - 16.637M in 5.082246s
Comparison:
Without: 3281310.6 i/s
With: 21856.1 i/s - 150.13x slower
Ruby 3.3.3 Preview 2 (ruby 3.3.0preview2 (2023-09-14 master e50fcca9a7) +YJIT [arm64-darwin22.6.0])
Warming up --------------------------------------
With 1.044k i/100ms
Without 288.264k i/100ms
Calculating -------------------------------------
With 24.044k (±192.0%) i/s - 29.232k in 5.305651s
Without 3.096M (± 8.0%) i/s - 15.566M in 5.059470s
Comparison:
Without: 3095640.6 i/s
With: 24043.8 i/s - 128.75x slower
Notes¶
Here's the environment I'm using:
- macOS: 13.6.1, Apple M1
Updated by ahorek (Pavel Rosický) 10 months ago
By fixing this, we could improve refinement performance and encourage more adoption since refinements are a powerful way to make small adjustments to the language without resorting to more complicated solutions -- or worse -- using monkey patches.
this shouldn't really affect the runtime performance of real programs unless you're dynamically creating refined modules like in your benchmark (which is also slow). Do you have an example where this could be a problem?
Updated by bkuhlmann (Brooke Kuhlmann) 10 months ago
this shouldn't really affect the runtime performance of real programs
Thanks and fair point. I'm using the above benchmark to track performance across Ruby versions. I agree, dynamically creating refinements isn't normal but wanted to simulate the defining of a refinement. I know there is cost in defining the module each time but am trying to zero in on the use of refine
, specifically. Maybe this is the wrong way to tackle the problem and I should be using a different benchmark?
Do you have an example where this could be a problem?
I'm afraid not. Well, not as easily encapsulated, that is. I do have the following:
- Benchmarks: This is an attempt to benchmark refinement performance from multiple angles (along with other non-refinement based benchmarks).
- Refinements: The gem -- which I maintain -- and have been using in multiple projects. Also the impetus for wanting to investigate the cost of defining refinements.
Updated by Eregon (Benoit Daloze) 9 months ago
Maybe this is the wrong way to tackle the problem and I should be using a different benchmark?
Yes, the performance of refine
itself should be irrelevant, it's paid once during startup (unless it would amount to taking a significant overhead to load a real app).
Just like calling Module.new
in the inner loop of an app is a mistake, those tools (module definition, defining refinements) are meant to be used during code loading and very little after.
Updated by bkuhlmann (Brooke Kuhlmann) 9 months ago
OK, thanks. Make sense. Feel free to close this.
Updated by jeremyevans0 (Jeremy Evans) 9 months ago
- Status changed from Open to Closed