Feature #18351: Support anonymous rest and keyword rest argument forwarding

• Status: Closed
• Assignee: matz (Yukihiro Matsumoto)
• Target version: -

[jeremyevans0 (Jeremy Evans)]

I would like to add support for the following syntax:

def foo(*)
  bar(*)
end
def baz(**)
  quux(**)
end

This is a natural addition after the introduction of anonymous block forwarding. Anonymous rest and keyword rest arguments were already supported in method parameters, this just allows them to be used as arguments to other methods. The same advantages of anonymous block forwarding apply to rest and keyword rest argument forwarding.

I've submitted a pull request implementing this syntax: https://github.com/ruby/ruby/pull/5148

[Eregon (Benoit Daloze)]

Same comment as on https://github.com/ruby/ruby/pull/4961 (I'll copy for convenience):

This could be problematic, because in various discussions and notably https://bugs.ruby-lang.org/issues/18011#note-6 and https://bugs.ruby-lang.org/issues/16456#note-9 it was relied that only ... produces [:rest, :*]:
Currently:

$ ruby -e 'p method(def m(...); end).parameters'
[[:rest, :*], [:block, :&]]
$ ruby -e 'p method(def m(*); end).parameters'
[[:rest]]

I think we should not change the result of parameters for these cases, otherwise we may have compatibility issues and we won't have a reliable way to detect ....

[ko1 (Koichi Sasada)]

compare with ... parameter delegation, the advantage is we can specify with * and ** respectively, like def foo(*, k1:nil); bar(*, k2: k1); end ?

[mame (Yusuke Endoh)]

@matz (Yukihiro Matsumoto) basically accepted this proposal for Ruby 3.2.

The concern about Method#parameters should be discussed after Ruby 3.1 is released. Now Method#parameters of def foo(...) returns [[:rest, :*], [:keyrest, :**], [:block, :&]]. After this change, perhaps it would be unable to distinguish between def foo(...) and def foo(*, **, &). But, I have no idea whether we should be able to distinguish them.

[jeremyevans0 (Jeremy Evans)]

mame (Yusuke Endoh) wrote in #note-3:

The concern about Method#parameters should be discussed after Ruby 3.1 is released. Now Method#parameters of def foo(...) returns [[:rest, :*], [:keyrest, :**], [:block, :&]]. After this change, perhaps it would be unable to distinguish between def foo(...) and def foo(*, **, &). But, I have no idea whether we should be able to distinguish them.

I don't think there is a need to distinguish them. In both the foo(...) and def foo(*, **, &) cases, the method accepts the same parameters (all normal parameters, all keyword parameters, and a block). You have no idea what the method is doing with the arguments. In both cases, the method can ignore the arguments, or delegate them. foo(*, **, &) is more flexible in that it can delegate the arguments separately, which foo(...) cannot do, but that's an implementation detail of the method, not what arguments it accepts (which is what Method#parameters returns).

It is possible to choose different names for the * and ** anonymous parameters, and recognize those names and keep Method#parameters output the same as it currently is, but I think the behavior in the current PR is better. Can anyone show a problem with having Method#parameters for foo(...) and def foo(*, **, &) return the same result?

[Eregon (Benoit Daloze)]

jeremyevans0 (Jeremy Evans) wrote in #note-4:

Can anyone show a problem with having Method#parameters for foo(...) and def foo(*, **, &) return the same result?

I think there may be value to distinguish ... and *, **, &.
That's pretty much the subject of #16456.
For instance a method taking ... seems to very very likely use delegation, while a method using *, **, & not necessarily (e.g., it could pass positional to one methods, kwargs to another and block to a third method).

Another issue is if one wants to detect ... they would actually need to check parameters == [[:rest, :*], [:keyrest, :**], [:block, :&]] and not e.g. just parameters.include?([:rest, :*]) which would also be true for def m(*).

Also, this changes the parameters for def m(*) from [[:rest]] to [[:rest, :*]], I think that's bad as some code might rely on that.

[jeremyevans0 (Jeremy Evans)]

Eregon (Benoit Daloze) wrote in #note-5:

jeremyevans0 (Jeremy Evans) wrote in #note-4:

Can anyone show a problem with having Method#parameters for foo(...) and def foo(*, **, &) return the same result?

I think there may be value to distinguish ... and *, **, &.
That's pretty much the subject of #16456.
For instance a method taking ... seems to very very likely use delegation, while a method using *, **, & not necessarily (e.g., it could pass positional to one methods, kwargs to another and block to a third method).

The purpose of Method#parameters is to show what arguments are accepted by the method, not how the arguments are handled.

Another issue is if one wants to detect ... they would actually need to check parameters == [[:rest, :*], [:keyrest, :**], [:block, :&]] and not e.g. just parameters.include?([:rest, :*]) which would also be true for def m(*).

Since ... and *, **, & accept the same arguments, from Method#parameters perspective, treating them the same seems reasonable.

Also, this changes the parameters for def m(*) from [[:rest]] to [[:rest, :*]], I think that's bad as some code might rely on that.

You are correct that callers might need to make changes, but they should be used to that. Callers already need to handle [:rest, :*] if they are handling .... They needed new handling in 2.7 for ... and **nil, and will need new handling in 3.1 for the change in ... (as 3.1 includes :keyrest for ...). Basically, the Method#parameters return value has been changing regularly, so this change seems like it should be acceptable. That being said, it's possible to use a different value for the internal local variable for * and **, and recognize it in Method#parameters, to keep the output the same.

[Eregon (Benoit Daloze)]

All good points, I think it's OK to use the current names in the PR and experiment with it.
If we see too many breaking changes it might be worth trying to keep them without a name from the #parameters point of view.

[palkan (Vladimir Dementyev)]

@jeremyevans0 (Jeremy Evans) I've been working on supporting this feature for Ruby Next, and found a weird edge case:

def foo_with_pattern(*, **)
  case some_value
  in [0, 1, *]
    bar(*)
  in {a:, b:, **}
    baz(**)
  end
end

As far as I understand, the delegation works here; and confusion is present, too.

[jeremyevans0 (Jeremy Evans)]

palkan (Vladimir Dementyev) wrote in #note-8:

@jeremyevans0 (Jeremy Evans) I've been working on supporting this feature for Ruby Next, and found a weird edge case:

def foo_with_pattern(*, **)
  case some_value
  in [0, 1, *]
    bar(*)
  in {a:, b:, **}
    baz(**)
  end
end

As far as I understand, the delegation works here; and confusion is present, too.

My expected behavior is that bar(*) and baz(**) use the arguments to foo_with_pattern, not the values in the pattern match, because * and ** in a pattern match are for ignoring arguments, not for capturing (* and ** as parameters have always captured to support super). I tested it and that is how it behaves, so the behavior matches my expectation.

However, my expectation isn't all that important, @matz's expectation is, so hopefully he can respond with how he would like it to behave. I plan on merging the support fairly soon, but we have a lot of time to modify the behavior before the release of Ruby 3.2, should @matz (Yukihiro Matsumoto) desire different behavior.

• Status changed from Open to Closed

Applied in changeset git|f53dfab95c30e222f67e610234f63d3e9189234d.

---

Add support for anonymous rest and keyword rest argument forwarding

This allows for the following syntax:

def foo(*)
  bar(*)
end
def baz(**)
  quux(**)
end

This is a natural addition after the introduction of anonymous
block forwarding. Anonymous rest and keyword rest arguments were
already supported in method parameters, this just allows them to
be used as arguments to other methods. The same advantages of
anonymous block forwarding apply to rest and keyword rest argument
forwarding.

This has some minor changes to #parameters output. Now, instead
of [:rest], [:keyrest], you get [:rest, :*], [:keyrest, :**].
These were already used for ... forwarding, so I think it makes
it more consistent to include them in other cases. If we want to
use [:rest], [:keyrest] in both cases, that is also possible.

I don't think the previous behavior of [:rest], [:keyrest] in
the non-... case and [:rest, :*], [:keyrest, :**] in the ...
case makes sense, but if we did want that behavior, we'll have to
make more substantial changes, such as using a different ID in the
... forwarding case.

Implements [Feature #18351]

[nobu (Nobuyoshi Nakada)]

• Related to Bug #18828: [Ripper] Anonymous parameter forwarding failures are not checked added