Project

General

Profile

Actions

Bug #16840

closed

Decrease in Hash#[]= performance with object keys

Added by ana06 (Ana Maria Martinez Gomez) over 4 years ago. Updated over 3 years ago.

Status:
Closed
Assignee:
-
Target version:
-
[ruby-core:98197]

Description

I was playing around with Ruby hashing and I have discovered something strange/surprising.

The file test.rb looks like:


require 'benchmark'
$N = 100000

class Ana 
end

objects = Array.new($N) { Ana.new() }
hash = {}
puts Benchmark.measure { 100.times { objects.each { |obj| hash[obj] = true } }}

I executed test.rb with different Ruby versions and it takes longer with newer versions. There is 1.5 seconds difference between Ruby 2.5 and master. Is that expected? And if so, why? Those are the execution results:

> rbenv shell 2.5.0
> ruby -v
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux]
> ruby test.rb
  2.236504   0.003546   2.240050 (  2.240256)
> ruby test.rb
  2.247041   0.003680   2.250721 (  2.250860)
> ruby test.rb
  2.276305   0.000351   2.276656 (  2.276829)
>·
> rbenv shell 2.6.2
> ruby -v
ruby 2.6.2p47 (2019-03-13 revision 67232) [x86_64-linux]
> ruby test.rb
  2.579052   0.004181   2.583233 (  2.583541)
> ruby test.rb
  2.580179   0.000000   2.580179 (  2.580362)
> ruby test.rb
  2.646516   0.000441   2.646957 (  2.647398)
>·
> rbenv shell 2.7.1
> ruby -v
ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux]
> ruby test.rb
  2.915415   0.004209   2.919624 (  2.920206)
> ruby test.rb
  2.867767   0.007511   2.875278 (  2.875416)
> ruby test.rb
  2.877741   0.000410   2.878151 (  2.878431)
>·
> rbenv shell 2.8.0-dev
> ruby -v
ruby 2.8.0dev (2020-05-07T16:22:38Z master 7ded8fd29a) [x86_64-linux]
> ruby test.rb
  3.840961   0.007852   3.848813 (  3.849499)
> ruby test.rb
  3.748391   0.007833   3.756224 (  3.756520)
> ruby test.rb
  3.686487   0.001656   3.688143 (  3.688332)

Related issues 1 (0 open1 closed)

Related to Ruby master - Feature #16837: Can we make Ruby 3.0 as fast as Ruby 2.7 with the new assertions?ClosedActions
Actions #1

Updated by k0kubun (Takashi Kokubun) over 4 years ago

  • Related to Feature #16837: Can we make Ruby 3.0 as fast as Ruby 2.7 with the new assertions? added

Updated by jeremyevans0 (Jeremy Evans) over 4 years ago

  • Subject changed from Why is Ruby getting slower in every version? to Decrease in Hash#[]= performance with object keys

Running your example code, I see a speedup between 2.5 and 2.6, but a slowdown in 2.7 and in master.

ruby 2.5.8p224 (2020-03-31 revision 67882) [x86_64-openbsd]
 10.040000   0.010000  10.050000 ( 10.089013)
  9.930000   0.010000   9.940000 ( 10.060374)
  9.850000   0.010000   9.860000 ( 10.048056)

ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-openbsd]
  9.450000   0.020000   9.470000 (  9.588573)
  9.470000   0.010000   9.480000 (  9.582035)
  9.500000   0.010000   9.510000 (  9.607103)

ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-openbsd]
 10.610000   0.010000  10.620000 ( 10.825456)
 10.660000   0.030000  10.690000 ( 10.845059)
 10.670000   0.000000  10.670000 ( 10.980962)

# Not sure if compile flags are exactly the same as the above
ruby 2.8.0dev (2020-05-19) [x86_64-openbsd6.7]
 11.670000   0.010000  11.680000 ( 11.735787)
 11.380000   0.040000  11.420000 ( 11.631791)
 11.410000   0.010000  11.420000 ( 11.677220)

These numbers show a ~20% performance decrease in a specific microbenchmark (Hash#[]= using objects as keys) between 2.6 and master. The difference between 2.6 and 2.7 could be related to #object_id changes. A similar benchmark with string keys shows no performance decrease between 2.6 and 2.7, for example. The difference between 2.7 and master could be related to the new assertions, which will hopefully be compiled out before release.

The percentage difference between versions in your benchmark output is much higher. It would be a good idea to get output from more people. The percentage difference may vary widely depending on hardware, compiler, and compilation flags used.

Updated by jeremyevans0 (Jeremy Evans) over 3 years ago

  • Status changed from Open to Closed

Apparently this case was optimized in Ruby 3.0. Using the benchmark code provided:

$ for x in 25 26 27 30; do ruby$x -v; for y in 1 2 3; do ruby$x t/t51.rb; done; done
ruby 2.5.9p229 (2021-04-05 revision 67939) [x86_64-openbsd]
 10.040000   0.010000  10.050000 ( 10.045536)
  9.970000   0.010000   9.980000 (  9.995454)
 10.170000   0.000000  10.170000 ( 10.180147)
ruby 2.6.7p197 (2021-04-05 revision 67941) [x86_64-openbsd]
  9.340000   0.000000   9.340000 (  9.326443)
  9.420000   0.010000   9.430000 (  9.425656)
  9.300000   0.020000   9.320000 (  9.327280)
ruby 2.7.3p183 (2021-04-05 revision 6847ee089d) [x86_64-openbsd]
 10.790000   0.020000  10.810000 ( 10.809778)
 10.840000   0.020000  10.860000 ( 10.866930)
 10.820000   0.010000  10.830000 ( 10.852136)
ruby 3.0.1p64 (2021-04-05 revision 0fb782ee38) [x86_64-openbsd]
  4.780000   0.020000   4.800000 (  4.798934)
  4.760000   0.020000   4.780000 (  4.770590)
  4.720000   0.050000   4.770000 (  4.774958)

Possibly due to 5f6053824551aec947a1c53d08975595aca1e513, which changed from an else if chain to a switch, but I didn't confirm that. Anyway, since it's now much faster than before, I think this can be closed.

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0