Feature #17785: Allow named parameters to be keywords

• Status: Assigned
• Assignee: matz (Yukihiro Matsumoto)
• Target version: -

[marcandre (Marc-Andre Lafortune)]

We should allow named parameters to be keywords and use add a trailing _ to the corresponding variable:

def check(arg, class:)
  arg.is_a?(class_)
end

check(42, class: Integer) # => true

Currently, if we want such an API we have to use **rest:

def check(arg, **rest)
  class_ = rest.fetch(:class) { raise ArgumentError('missing keyword: :class')}
  if rest.size > 1
    unknown = rest.keys - [:class]
    raise ArgumentError("unknown keyword(s): :#{unknown.join(', :')})
  end

  arg.is_a?(class_)
end

This is very verbose, much less convenient, much less readable, prevents steep from generating the proper signature, etc.

We should do the same for pattern match.

[mame (Yusuke Endoh)]

An interesting idea. I have never thought of it. A clearer name might be better, such as keyword_variable_class, instead of class_.

FYI, Binding#local_variable_get was introduced for the very use case.

def check(arg, class:)
  class_ = binding.local_variable_get(:class)
  arg.is_a?(class_)
end

Personally I don't like local_variable_get, though, because it is unreasonably slow, and still less convenient.

[zverok (Victor Shepelev)]

We actually can:

def check(arg, class:)
  arg.is_a?(binding.local_variable_get('class'))
end

Here @nobu (Nobuyoshi Nakada) have argued that this is exactly how it is intended to be done.

I vaguely remember arguing somewhere about some syntax to access "specially named" local vars, but I don't remember what I've proposed then, and can't find the ticket :shrug:

[marcandre (Marc-Andre Lafortune)]

Clearly, class_ is much simpler and much faster than binding.local_variable_get(:class)...

[byroot (Jean Boussier)]

Arguably it's a bit of a stretch, but how would you handle: foo(class_, class:)?

What if instead of mangling the variable name, there would be a way to tell the parser to interpret the next word as a regular name rather than a keyword? e.g.:

def check(arg, class:)
  arg.is_a?(\class)
end

\ being a common escaping character I think it the one that would make the most sense.

And that would allow to make it work with regular parameters as well:

def diff(start, \end)
  \end - start
end

Even though this use case is much less important, except for documentation purposes.

[Eregon (Benoit Daloze)]

I like @byroot's idea to solve the more general issue and not just this specific instance.

marcandre (Marc-Andre Lafortune) wrote in #note-3:

Clearly, class_ is much simpler and much faster than binding.local_variable_get(:class)...

It can be the same performance with a JIT and escape analysis (i.e., it's the same on TruffleRuby).

[austin (Austin Ziegler)]

I’ll also say I like byroot’s idea, especially as bare \VALUE
currently throws a SyntaxError.

[duerst (Martin Dürst)]

I think it's not a good idea to introduce special syntax such as class_ just for the case where arguments are named with keywords. First, the number of keywords is very low, which means that the cases where using a keyword as an argument name makes sense is also very low. Second, there are keywords such as if and else that are of very doubtful use as variable names anyway. Third, using keywords as variable names inherently increases the cognitive load on the reader and is a source for confusion.

Also, the special meaning of the trailing underscore will be difficult to recognize and understand for most people because it will appear so rarely. And the _ doesn't match well with the : in the argument list. And _ also is already allowed, so at least in theory, there's a chance of compatibility problems.

And then there's good old klass, which did the job for decades. And for those who don't like klass, there's class_. What's the problem of using class_ in the argument list if your plan is to use it in the body of the method anyway?

marcandre (Marc-Andre Lafortune) wrote in #note-3:

Clearly, class_ is much simpler and much faster than binding.local_variable_get(:class)...

What about finding something in between the two? E.g. even just introducing variable_get as an alias to binding.local_variable_get would make this easier to use. And if this really needs optimization, it could be done, too, but using a different argument name would solve the problem.

With respect to \, it reminds me of older languages (such as m4, C, and TeX) where there's a purely string-based level below (or before) the usual structured syntax. Do we want Ruby to descend to that level? Escaping exists inside strings because you don't want the range of data you can handle with a programming language to be restricted by the syntax of the language itself. Also, escaping inside strings is frequent enough for everybody, and occurs in a very similar form across a wide range of programming languages, so that every programmer knows it. Backslashes in front of keywords would be a whole different matter.

There are programming languages where there are no reserved keywords. The one I know and have used is PL/1. If not having any keywords would have been a design goal of Ruby, I'm sure Matz would have found a way to get there. But it wasn't, and I guess it isn't. And as far as I understand, this proposal doesn't get us there.

In conclusion, I think this issue chases a phantom. The trade-off (rarely used obscure syntax to solve a rarely occurring pseudo-problem) is not good. It would introduce some very rarely used edge-case syntax, and wouldn't really make the language any better.

If there are no more urgent kinds of improvements to Ruby syntax that this one, then we know Ruby is in a pretty good place!

[nobu (Nobuyoshi Nakada)]

duerst (Martin Dürst) wrote in #note-7:

What about finding something in between the two? E.g. even just introducing variable_get as an alias to binding.local_variable_get would make this easier to use. And if this really needs optimization, it could be done, too, but using a different argument name would solve the problem.

In built-in methods written in Ruby, we chose __builtin.arg!(:in) form (see timev.rb).

With respect to \, it reminds me of older languages (such as m4, C, and TeX) where there's a purely string-based level below (or before) the usual structured syntax. Do we want Ruby to descend to that level? Escaping exists inside strings because you don't want the range of data you can handle with a programming language to be restricted by the syntax of the language itself. Also, escaping inside strings is frequent enough for everybody, and occurs in a very similar form across a wide range of programming languages, so that every programmer knows it. Backslashes in front of keywords would be a whole different matter.

Agree, and backslashes will be troublesome in eval obviously.

[byroot (Jean Boussier)]

the number of keywords is very low, which means that the cases where using a keyword as an argument name makes sense is also very low.

Variable and keyword names are not purely random though, so I don't think this statistical reasoning makes that much sense. Especially since keywords reserved names that are short and popular: end, class, etc. If you define method that generate some HTML, a class: named parameters is common, if you define a method that deal with period of times, end: is common, if: is common for methods taking callbacks, etc.

I'm not for adding extra syntax, but I agree with @marcandree that binding.local_variable_get(:class) is too slow to be used in many cases.

It would be great if the parser or VM would just optimize it away, but I understand that it's currently very tricky because both #binding and Binding#local_variable_get could have been redefined.

[marcandre (Marc-Andre Lafortune)]

My main objection to local_variable_get is that it's super verbose / ugly.

How would you handle foo(class_, class:)?

Setting local class_ would not happen here. It would only happen if not shadowing an existing variable (except maybe if that other variable was also created the same way)

I like \class too.

[matheusrich (Matheus Richard)]

Since we have __method__, maybe adding something like __params__?

def check(arg, class:)
  arg.is_a?(__params__[:class])
end

check(42, class: Integer) # => true

Edit:

We would have to deal with the case where positional and keyword params have the same name too.

[nobu (Nobuyoshi Nakada)]

• Related to Feature #13207: Allow keyword local variable names like `class` or `for` added

[hibachrach (Hazel Bachrach)]

This feels related to this proposal I submitted ~1.5 years ago concerning external/internal names for keyword parameters as it would solve this problem somewhat more elegantly:

# Only one possible syntax--see above proposal for alternatives
def check(arg, class class_:)
  arg.is_a?(class_)
end

check(42, class: Integer) # => true

[ioquatix (Samuel Williams)]

I personally like \class too. I run into the issue from time to time.

[byroot (Jean Boussier)]

• Related to Feature #18402: Argument Labels added

[Dan0042 (Daniel DeLorme)]

matheusrich (Matheus Richard) wrote in #note-11:

Since we have __method__, maybe adding something like __params__?

I really like this idea. Although I would prefer having different names for positional vs keywords, so maybe __args__ and __kwargs__
This was previously suggested in #15049 (for the purpose of forwarding all keyword arguments) but it's very well suited to handling special keywords like if/for/class, without introducing extra syntax or special cases.

[Eregon (Benoit Daloze)]

matheusrich (Matheus Richard) wrote in #note-11:

Since we have __method__, maybe adding something like __params__?

I dislike this approach because it will introduce lots of complexity for Ruby implementations, and will likely make the language slower because arguments need to be retained longer than without it.
In the worst case it could even introduce non-obvious memory leaks (because one cannot know if __params__ would be used, potentially in an eval or aliases or so).

JavaScript's arguments is a well known PITA for implementations and often seen as hurting optimizations.

[matheusrich (Matheus Richard)]

Eregon (Benoit Daloze) wrote in #note-17:

matheusrich (Matheus Richard) wrote in #note-11:

Since we have __method__, maybe adding something like __params__?

I dislike this approach because it will introduce lots of complexity for Ruby implementations, and will likely make the language slower because arguments need to be retained longer than without it.
In the worst case it could even introduce non-obvious memory leaks (because one cannot know if __params__ would be used, potentially in an eval or aliases or so).

JavaScript's arguments is a well known PITA for implementations and often seen as hurting optimizations.

Yeah, that's really not ideal. Do you think a special syntax (like \class, for example) would be less complex for Ruby implementations?

[Dan0042 (Daniel DeLorme)]

Eregon (Benoit Daloze) wrote in #note-17:

because one cannot know if __params__ would be used, potentially in an eval or aliases or so

I agree, with that kind of complexity it wouldn't make sense. But I wasn't thinking of anything so complicated. If the token is lexically present in the method body, assign it a hash of the keyword arguments, just like a local variable. The allocation/cost is only for methods that use it. eval("__params__") is simply not supported. To me that's perfectly fine. Sort of like how eval("v=42"); v results in undefined v error. In all honesty I can't understand why anyone would want to support such an edge case of eval when it results in so much complexity that is simply not needed for the normal case.

[Eregon (Benoit Daloze)]

matheusrich (Matheus Richard) wrote in #note-18:

Eregon (Benoit Daloze) wrote in #note-17:
Yeah, that's really not ideal. Do you think a special syntax (like \class, for example) would be less complex for Ruby implementations?

Yes, then it's only complexity in the lexer and everything else would work unchanged.
The Oz language for instance has 'reserved' which works like \class above.
It's also much simpler conceptually, just help the lexer understand what you want, then everything keeps working as before.
The parser doesn't even need to know about it, it'll just see a identifier token instead of a keyword token.

Dan0042 (Daniel DeLorme) wrote in #note-19:

In all honesty I can't understand why anyone would want to support such an edge case of eval when it results in so much complexity that is simply not needed for the normal case.

Consistency and referential transparency (anything that works outside eval should work inside eval, true for most things in Ruby).

[Dan0042 (Daniel DeLorme)]

Dan0042 (Daniel DeLorme) wrote in #note-19:

Eregon (Benoit Daloze) wrote in #note-17:

because one cannot know if __params__ would be used, potentially in an eval or aliases or so

I agree, with that kind of complexity it wouldn't make sense.

Wait a sec... actually there's precedent for this. super does this. It forwards all arguments up the inheritance chain. And it's possible to do eval("super"). __params__ is like the first half of super; just collect the arguments, without the subsequent method call. So it's definitely possible (and already done), technically.

[Eregon (Benoit Daloze)]

Dan0042 (Daniel DeLorme) wrote in #note-21:

Wait a sec... actually there's precedent for this. super does this. It forwards all arguments up the inheritance chain. And it's possible to do eval("super"). __params__ is like the first half of super; just collect the arguments, without the subsequent method call. So it's definitely possible (and already done), technically.

super (or zsuper in parser terms) does rereads all arguments.
It's already a nightmare in terms of complexity FWIW.

But __params__(:name) is far worse, at least zsuper only builds an array not some magic mapping from variable name to value.

Ah and eval("super") should probably be deprecated, it seems such a bad idea and I doubt it works well on many Ruby impls.

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Assigned