Project

General

Profile

Bug #14015

Enumerable & Hash yielding arity

Added by marcandre (Marc-Andre Lafortune) over 1 year ago. Updated 10 months ago.

Status:
Open
Priority:
Normal
Target version:
-
ruby -v:
2.5.0 preview 1
[ruby-core:83281]

Description

The subtle difference between yield 1, 2 and yield [1, 2] has always confused me.

Today I wanted to pass a method to Hash#flat_map and realized how it's even more confusing than I thought.

I assumed that Hash#each was calling yield key, value. But somehow it's not that simple:

{a: 1}.map(&->(key, value){}) # => [nil]
{a: 1}.flat_map(&->(key, value){})  #=> ArgumentError: wrong number of arguments (given 1, expected 2)

What blows my mind, is that a custom method each that does yield a, 1 has different result!

class << o = Object.new
  include Enumerable
  def each
    yield :a, 1
  end
end
o.map(&->(key, value){})  # => [nil]
o.flat_map(&->(key, value){})  # => [nil]  does not raise!!

I don't even know how that's possible, since Hash doesn't have a specialized flat_map method...

Here's a list of methods that accept a lambda of arity 2 (as I would expect)
For Hash
each, any?, map, select, reject,
For a custom yield
each, any?, map, count, find_index, flat_map, all?, one?, none?, take_while, uniq

These two lists have each, map and any? in common. Others work in one flavor, not the other. Many require arity 1: find, sort_by, grep, grep_v, count, detect, find_index, find_all, ...

To make things even more impossible, Hash#map has been working with arity 2 since Ruby 2.4 only.

Finally, Hash#each changes the expected arity of select, reject, and any?, but not of map:

    {a: 1}         .select(&->(a, b){})  # => {}
    {a: 1}.each.select(&->(a, b){}) # => wrong number of arguments (given 1, expected 2)

Conclusion:

It seems more or less impossible to guess the expected arity of methods of Enumerable and of Hash, and they are not even consistent with one another. This makes these methods more or less unusable with lambdas.

While compatibility could be an issue, the fact that Hash#map has changed it's arity (I believe following https://bugs.ruby-lang.org/issues/13391 ) makes me think that compatibility with the lesser used methods would be even less of a problem.

My personal wish: that the following methods be fixed to expect arity 2 for lambdas:

For both Hash & Enumerable:

  • find, sort_by, grep, grep_v, detect, find_all, partition, group_by, min_by, max_by, minmax_by, reverse_each, drop_while, sum For Hash:
  • count, find_index, flat_map, all?, one?, none?, take_while, uniq For Enumerable:
  • select, reject

Matz, what do you think?


Files

yield_arity.rb (805 Bytes) yield_arity.rb marcandre (Marc-Andre Lafortune), 10/14/2017 08:25 PM

History

Updated by marcandre (Marc-Andre Lafortune) over 1 year ago

I'm attaching a script I used to test this, in case it can be helpful

Updated by nobu (Nobuyoshi Nakada) over 1 year ago

  • Description updated (diff)

We know, but can't fix for backward compatibilities, now.

Updated by marcandre (Marc-Andre Lafortune) over 1 year ago

Reading more, I see that in some cases there were differences between a lambda with arity 2 and a method with arity 2.
In particular, the change of Hash#map in Ruby 2.4.2 was only for methods, but lambdas were working before. What a mess.
Any hope to have all of that resolved at some point?

Updated by shevegen (Robert A. Heiler) over 1 year ago

Perhaps ruby 3.x?

To be honest, while I agree with Marc here, if only for consistency, I
think it is probably not the biggest issue overall. I use yield a lot
but most of my use cases are very simple with yield. Not that I am saying
to be representative of any real use case, either, though. :)

I guess it may be harder and more confusing for people who use lambdas
a lot.

Updated by marcandre (Marc-Andre Lafortune) over 1 year ago

Would it be possible to issue a warning for cases of lambdas with arity 1 used in 2.5, so we can go forward in the next version?

Updated by marcandre (Marc-Andre Lafortune) over 1 year ago

Matz, this was on the agenda for the developers meeting, was it discussed?

Updated by matz (Yukihiro Matsumoto) over 1 year ago

Hi,

I admit inconsistency exists. The behavior has changed time to time.
I will make the compromise to minimize the confusion. But we need some time.

Matz.

Updated by marcandre (Marc-Andre Lafortune) 10 months ago

Here's more code to show how the situation is complicated.
I don't know which of these can be considered bugs and which are as per spec.
The method check below checks if a method accepting one argument (one) or two arguments (two) is acceptable for a particular call.

def check(receiver, method, expects)
  results = expects.keys.map do |block|
    receiver.send(method, &block)
    :ok
  rescue ArgumentError
    :raise
  end
  raise "Expected #{expects.values}, got #{results}" unless expects.values == results
end

hash = {a: 1}

def method_one(x)
end
one = method(:method_one)

def method_two(x, y)
end
two = method(:method_two)

class << enum = Object.new
  include Enumerable
  def each
    return to_enum unless block_given?
    yield :a, 1
  end
end

### 1) Lambda vs Method

# Don't always behave the same way:

check(enum, :select,
  one           => :ok,
  (->(x) {})    => :ok,
  two           => :raise,
  (->(x, y) {}) => :ok,
) {|x| enum.select(&x) }

# (Since 2.2, https://bugs.ruby-lang.org/issues/9605)

# ### 2) Hash vs Enumerable

# Sometimes requires arity 1:
check(hash, :detect,
  one           => :ok,
  two           => :raise,
)
check(enum, :detect,
  one           => :ok,
  two           => :raise,
)


# Sometimes relaxed for Hash, but not Enumerable
check(hash, :any?,
  one           => :ok,
  two           => :ok,
)
check(enum, :any?,
  one           => :raise,
  two           => :ok,
)
# (But Hash#each no longer relaxed, now reversed from Enumerable)
check(hash.each, :any?,
  one           => :ok,
  two           => :raise,
)


# Sometimes requires arity 2 for Hash vs 1 for Enumerable
check(hash, :select,
  one           => :raise,
  two           => :ok,
)
check(enum, :select,
  one           => :ok,
  two           => :raise,
)
# (But Hash#each reverses:)
check(hash.each, :select,
  one           => :ok,
  two           => :raise,
)


# Sometimes requires the reverse: arity 1 for Hash vs 2 for Enumerable
check(hash, :find_index,
  one           => :ok,
  two           => :raise,
)
check(enum, :find_index,
  one           => :raise,
  two           => :ok,
)

# But Hash#each doesn't really behave as Enumerable...
check(hash.each, :find_index,
  one           => :ok,
  two           => :raise,
)

Ideally, we would have a decision as to what the correct behavior is, before we introduce a shorthand to get a method.

Also available in: Atom PDF