Bug #14015
closedEnumerable & Hash yielding arity
Description
The subtle difference between yield 1, 2
and yield [1, 2]
has always confused me.
Today I wanted to pass a method to Hash#flat_map and realized how it's even more confusing than I thought.
I assumed that Hash#each
was calling yield key, value
. But somehow it's not that simple:
{a: 1}.map(&->(key, value){}) # => [nil]
{a: 1}.flat_map(&->(key, value){}) #=> ArgumentError: wrong number of arguments (given 1, expected 2)
What blows my mind, is that a custom method each
that does yield a, 1
has different result!
class << o = Object.new
include Enumerable
def each
yield :a, 1
end
end
o.map(&->(key, value){}) # => [nil]
o.flat_map(&->(key, value){}) # => [nil] does not raise!!
I don't even know how that's possible, since Hash doesn't have a specialized flat_map
method...
Here's a list of methods that accept a lambda of arity 2 (as I would expect)
For Hash
each, any?, map, select, reject,
For a custom yield
each, any?, map, count, find_index, flat_map, all?, one?, none?, take_while, uniq
These two lists have each
, map
and any?
in common. Others work in one flavor, not the other. Many require arity 1: find, sort_by, grep, grep_v, count, detect, find_index, find_all, ...
To make things even more impossible, Hash#map
has been working with arity 2 since Ruby 2.4 only.
Finally, Hash#each
changes the expected arity of select
, reject
, and any?
, but not of map
:
{a: 1} .select(&->(a, b){}) # => {}
{a: 1}.each.select(&->(a, b){}) # => wrong number of arguments (given 1, expected 2)
Conclusion:
It seems more or less impossible to guess the expected arity of methods of Enumerable and of Hash, and they are not even consistent with one another. This makes these methods more or less unusable with lambdas.
While compatibility could be an issue, the fact that Hash#map
has changed it's arity (I believe following https://bugs.ruby-lang.org/issues/13391 ) makes me think that compatibility with the lesser used methods would be even less of a problem.
My personal wish: that the following methods be fixed to expect arity 2 for lambdas:
For both Hash & Enumerable:
- find, sort_by, grep, grep_v, detect, find_all, partition, group_by, min_by, max_by, minmax_by, reverse_each, drop_while, sum
For Hash: - count, find_index, flat_map, all?, one?, none?, take_while, uniq
For Enumerable: - select, reject
Matz, what do you think?
Files
Updated by marcandre (Marc-Andre Lafortune) almost 7 years ago
- File yield_arity.rb yield_arity.rb added
I'm attaching a script I used to test this, in case it can be helpful
Updated by nobu (Nobuyoshi Nakada) almost 7 years ago
- Description updated (diff)
We know, but can't fix for backward compatibilities, now.
Updated by marcandre (Marc-Andre Lafortune) almost 7 years ago
Reading more, I see that in some cases there were differences between a lambda with arity 2 and a method with arity 2.
In particular, the change of Hash#map
in Ruby 2.4.2 was only for methods, but lambdas were working before. What a mess.
Any hope to have all of that resolved at some point?
Updated by shevegen (Robert A. Heiler) almost 7 years ago
Perhaps ruby 3.x?
To be honest, while I agree with Marc here, if only for consistency, I
think it is probably not the biggest issue overall. I use yield a lot
but most of my use cases are very simple with yield. Not that I am saying
to be representative of any real use case, either, though. :)
I guess it may be harder and more confusing for people who use lambdas
a lot.
Updated by marcandre (Marc-Andre Lafortune) almost 7 years ago
Would it be possible to issue a warning for cases of lambdas with arity 1 used in 2.5, so we can go forward in the next version?
Updated by marcandre (Marc-Andre Lafortune) over 6 years ago
Matz, this was on the agenda for the developers meeting, was it discussed?
Updated by matz (Yukihiro Matsumoto) over 6 years ago
Hi,
I admit inconsistency exists. The behavior has changed time to time.
I will make the compromise to minimize the confusion. But we need some time.
Matz.
Updated by marcandre (Marc-Andre Lafortune) about 6 years ago
Here's more code to show how the situation is complicated.
I don't know which of these can be considered bugs and which are as per spec.
The method check
below checks if a method accepting one argument (one
) or two arguments (two
) is acceptable for a particular call.
def check(receiver, method, expects)
results = expects.keys.map do |block|
receiver.send(method, &block)
:ok
rescue ArgumentError
:raise
end
raise "Expected #{expects.values}, got #{results}" unless expects.values == results
end
hash = {a: 1}
def method_one(x)
end
one = method(:method_one)
def method_two(x, y)
end
two = method(:method_two)
class << enum = Object.new
include Enumerable
def each
return to_enum unless block_given?
yield :a, 1
end
end
### 1) Lambda vs Method
# Don't always behave the same way:
check(enum, :select,
one => :ok,
(->(x) {}) => :ok,
two => :raise,
(->(x, y) {}) => :ok,
) {|x| enum.select(&x) }
# (Since 2.2, https://bugs.ruby-lang.org/issues/9605)
# ### 2) Hash vs Enumerable
# Sometimes requires arity 1:
check(hash, :detect,
one => :ok,
two => :raise,
)
check(enum, :detect,
one => :ok,
two => :raise,
)
# Sometimes relaxed for Hash, but not Enumerable
check(hash, :any?,
one => :ok,
two => :ok,
)
check(enum, :any?,
one => :raise,
two => :ok,
)
# (But Hash#each no longer relaxed, now reversed from Enumerable)
check(hash.each, :any?,
one => :ok,
two => :raise,
)
# Sometimes requires arity 2 for Hash vs 1 for Enumerable
check(hash, :select,
one => :raise,
two => :ok,
)
check(enum, :select,
one => :ok,
two => :raise,
)
# (But Hash#each reverses:)
check(hash.each, :select,
one => :ok,
two => :raise,
)
# Sometimes requires the reverse: arity 1 for Hash vs 2 for Enumerable
check(hash, :find_index,
one => :ok,
two => :raise,
)
check(enum, :find_index,
one => :raise,
two => :ok,
)
# But Hash#each doesn't really behave as Enumerable...
check(hash.each, :find_index,
one => :ok,
two => :raise,
)
Ideally, we would have a decision as to what the correct behavior is, before we introduce a shorthand to get a method.
Updated by boblail (Bob Lail) almost 4 years ago
Looks like this was resolved for map
specifically in #13391 (https://github.com/ruby/ruby/commit/1f67a3900fbd45482ed36ad3b148b321307c1576)
Here are other steps to reveal the inconsistency:
hash = { a: "my", b: "en" }
hash.map { |k,v| [k,v].join } # => ["amy", "ben"]
hash.map &proc { |k,v| [k,v].join } # => ["amy", "ben"]
hash.map &lambda { |k,v| [k,v].join } # => ["amy", "ben"]
hash.flat_map { |k,v| [k,v].join } # => ["amy", "ben"]
hash.flat_map &proc { |k,v| [k,v].join } # => ["amy", "ben"]
hash.flat_map &lambda { |k,v| [k,v].join } # => ArgumentError (wrong number of arguments (given 1, expected 2))
Updated by marcandre (Marc-Andre Lafortune) almost 4 years ago
- Status changed from Open to Closed
Now that Hash#each
has arity 1, I'll close this and open an update in #17197