Project

General

Profile

Actions

Bug #20853

closed

Hash key retrieval after Process.warmup

Added by keelerm84 (Matthew Keeler) 22 days ago. Updated 12 days ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x86_64-linux]
[ruby-core:119641]

Description

This was first reported as an issue against the LaunchDarkly SDK and Sidekiq SDKs as an issue in Ruby 3.3.1. I can still in Ruby 3.3.5.

Overview of behavior
The LaunchDarkly SDK maintains a hash of feature and segment information. This hash is indexed by 2 keys which are themselves hashes. They are defined as:

  FEATURES = {
    namespace: "features",
    priority: 1,
    get_dependency_keys: lambda { |flag| (flag[:prerequisites] || []).map { |p| p[:key] } },
  }.freeze

  SEGMENTS = {
    namespace: "segments",
    priority: 0,
  }.freeze

When running the attached script, we often see an error indicating that the flag is not found. If you comment out the Process.warmup line, this error will go away.

The require 'active_support/all' directive is not strictly necessary. Including it causes the failure to more frequently occur, presumably due to the increase work in the warmup phase.

Hash key and access
When debugging the LaunchDarkly code, I noticed some odd behavior. Even though the hash seems populated, the FEATURES key could not index into it correctly.

Below is a bit of code, modified from the SDK, to show how we are trying to interact with the hash. The puts are added to clarify the issue.

    def get(kind, key)
      @lock.with_read_lock do

        @items.keys.each do |k|
          puts "##### Does #{kind[:namespace]} match the key #{k}? #{k == kind} #{k.eql?(kind)} #{k.hash == kind.hash}"
        end
        puts "###### Does @items include it? #{@items.include?(kind)}"

        coll = @items[kind]
        f = coll.nil? ? nil : coll[key.to_sym]
        (f.nil? || f[:deleted]) ? nil : f
      end
    end

Running without Process.warmup

If we run the attached script without the Process.warmup call, we will see that the keys align with the FEATURES key we provided. Additionally, the hash does see that the key exists, resulting in an evaluation result of true at the end.

That output follows.

##### Does features match the key {:namespace=>"segments", :priority=>0}? false false false
##### Does features match the key {:namespace=>"features", :priority=>1, :get_dependency_keys=>#<Proc:0x00007ee6174f9738 /home/mkeeler/code/launchdarkly/ruby-server-sdk.git/main/lib/ldclient-rb/in_memory_store.rb:17 (lambda)>}? true true true
###### Does @items include it? true
true

Running with Process.warmup

If you enable the Process.warmup option, and run the script a few times, you will see output similar to the following.

Note that while the keys are ==, eql?, and their .hash values are equal, the hash does not see that the FEATURES key exists.

##### Does features match the key {:namespace=>"segments", :priority=>0}? false false false
##### Does features match the key {:namespace=>"features", :priority=>1, :get_dependency_keys=>#<Proc:0x0000774f1cbfd730 /home/mkeeler/code/launchdarkly/ruby-server-sdk.git/main/lib/ldclient-rb/in_memory_store.rb:17 (lambda)>}? true true true
###### Does @items include it? false
I, [2024-10-30T14:34:33.842145 #974568]  INFO -- : [LDClient] Unknown feature flag "sample-feature". Returning default value
false

Workarounds

We have worked around this issue by replacing these keys with classes. You can refer to the PR here for those details.

You can also resolve this by triggering a rehash on the hash after the Process.warmup has run.


Files

main.rb (630 Bytes) main.rb keelerm84 (Matthew Keeler), 10/30/2024 06:50 PM
Gemfile (96 Bytes) Gemfile keelerm84 (Matthew Keeler), 10/30/2024 06:50 PM

Updated by peterzhu2118 (Peter Zhu) 22 days ago

Thank you for reporting this bug.

I believe the issue here is that because Process.warmup runs a GC compaction, it can cause objects to move. While this is usually not a problem, Proc#hash is calculated using pointer values. Since the pointer values can change, the hash value can change as well. For example:

p = proc {}

puts p.hash # => 2411193347801829553

GC.verify_compaction_references(expand_heap: true, toward: :empty)

puts p.hash # => 1198623829435689723

This causes the hash value for the Hash to change as well:

hash = {}
key = {:namespace=>"features", :priority=>1, :get_dependency_keys=>proc {} }

hash[key] = 1

puts hash.key?(key) # => true

GC.verify_compaction_references(expand_heap: true, toward: :empty)

puts hash.key?(key) # => false

I'll work on a fix for this problem.

Actions #2

Updated by peterzhu2118 (Peter Zhu) 22 days ago

  • Backport changed from 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN to 3.1: UNKNOWN, 3.2: REQUIRED, 3.3: REQUIRED

Updated by peterzhu2118 (Peter Zhu) 22 days ago

I have a fix available here: https://github.com/ruby/ruby/pull/11966

Please let me know if this fixes your issues.

Updated by keelerm84 (Matthew Keeler) 22 days ago

Yes, that appears to have resolved the issue. Thank you for such a quick turnaround.

Actions #5

Updated by byroot (Jean Boussier) 22 days ago

  • Backport changed from 3.1: UNKNOWN, 3.2: REQUIRED, 3.3: REQUIRED to 3.1: WONTFIX, 3.2: REQUIRED, 3.3: REQUIRED
Actions #6

Updated by peterzhu2118 (Peter Zhu) 20 days ago

  • Status changed from Open to Closed

Applied in changeset git|29c480dd6fca993590c82078ba797e2c4e876ac7.


[Bug #20853] Fix Proc#hash to not change after compaction

The hash value of a Proc must remain constant after a compaction, otherwise
it may not work as the key in a hash table.

Updated by k0kubun (Takashi Kokubun) 17 days ago

  • Backport changed from 3.1: WONTFIX, 3.2: REQUIRED, 3.3: REQUIRED to 3.1: WONTFIX, 3.2: REQUIRED, 3.3: DONE

Updated by nagachika (Tomoyuki Chikanaga) 12 days ago

  • Backport changed from 3.1: WONTFIX, 3.2: REQUIRED, 3.3: DONE to 3.1: WONTFIX, 3.2: DONE, 3.3: DONE
Actions

Also available in: Atom PDF

Like0
Like1Like0Like1Like0Like0Like0Like0Like0