Feature #11816
openPartial safe navigation operator
Description
I'm extremely surprised (and disappointed) that, currently:
x = nil
x&.foo.bar # => NoMethodError: undefined method `bar' for nil:NilClass
To make it safe, you have to write x&.foo&.bar
. But if foo
is never supposed to return nil
, then that code isn't "fail early" in case it actually does. nil&.foo.bar
is more expressive, simpler and is perfect if you want to an error if foo
returned nil
. To actually get what you want, you have to resort using the old form x && x.foo.bar
...
In CoffeeScript, you can write x()?.foo.bar
and it will work well, since it gets compiled to
if ((_ref = x()) != null) {
_ref.foo.bar;
}
All the discussion in #11537 focuses on x&.foo&.bar
, so I have to ask:
Matz, what is your understanding of x&.foo.bar
?
I feel the current implementation is not useful and should be changed to what I had in mind. I can't see any legitimate use of x&.foo.bar
currently.
Updated by marcandre (Marc-Andre Lafortune) almost 9 years ago
BTW, this came up in Matt Larraz's PR https://github.com/ruby/ruby/pull/1142
Both Hiroshi Shibata and Benoit Daloze thought (as I did) that e_error_bytes&.dup.force_encoding(...)
should work.
Updated by nobu (Nobuyoshi Nakada) almost 9 years ago
I haven't thought it at all, but it seems interesting.
Updated by mame (Yusuke Endoh) almost 9 years ago
It seems very sensitive to determine how long it propagates.
How should we interpret str&.upcase + "foo"
?
If it is considered as str&.upcase&.+("foo")
, it may return nil (as I expect).
If it is considered as (str&.upcase) + "foo"
, it may raise an exception. We need write p str&.upcase&.+("foo")
.
How's that for str&.upcase == nil
?
If it is considered as str&.upcase&.==(nil)
, it will return false or nil. Very confusing.
If it is considered as (str&.upcase) == nil
, it will return true or false.
--
Yusuke Endoh mame@ruby-lang.org
Updated by nobu (Nobuyoshi Nakada) almost 9 years ago
If it propagates, we could write safe aref as:
ary&.itself[idx]
ary&.itself[idx] = value
Updated by matz (Yukihiro Matsumoto) almost 9 years ago
Marc,
We need more clarification. Did you mean all method calling dots should be interpreted as &.
?
What about other calls like array/hash indexing or operators?
Matz.
Updated by marcandre (Marc-Andre Lafortune) almost 9 years ago
Glad to see we're headed towards having x&.foo.bar
not be the same as (x&.foo).bar
.
When we ask about &.
vs operators, we can see that the question is similar to that of precedence, right? We could say that currently in trunk, &.
has same "precedence" as .
, which we now agree is not ideal. We need to bump &.
down at least one level.
So either &.
has a "precedence" just lower than .
, or we (well, Matz) decide to make it even lower.
My guess is that the ideal would if &.
has a "precedence" in between comparison operators (<, >=, ...) and equality operators (==, <=>, ...)
This way:
x = nil
x&.foo.bar # => nil
x&.foo[42] # => nil
x&.foo[42] = 43 # => nil
x&.foo * 42 # => nil
x&.foo + 42 # => nil
x&.foo << 42 # => nil
x&.foo < 42 # => nil
x&.foo == 42 # => false ### This is where the precedence of &. is higher
x&.foo || 42 # => 42
x&.foo ? 1 : 2 # => 2
My reasoning is that this divides the operators cleanly into those that we never want to apply to nil
(e.g. +
), from those that can be applied to it (e.g. ||
).
Updated by marcandre (Marc-Andre Lafortune) almost 9 years ago
Did you mean all method calling dots should be interpreted as
&.
?
Just to be clear, no, there should be a difference between x&.foo&.bar
and x&.foo.bar
. The following should be equivalent:
x&.foo.bar
# same as
if temp = x&.foo
temp.bar
end
In particular:
x = Object.new
def x.foo; nil; end
x&.foo&.bar # => nil
x&.foo.bar # => NoMethodError, no method :bar for nil
Updated by mlarraz (Matt Larraz) almost 9 years ago
My naïve understanding is that foo&.bar
should be a shorthand for foo && foo.bar
, and therefore the &.
operator should take the same level of precedence as the &&
operator.
Marc-Andre Lafortune wrote:
x = nil x&.foo.bar # => nil x&.foo[42] # => nil x&.foo[42] = 43 # => nil x&.foo * 42 # => nil x&.foo + 42 # => nil x&.foo << 42 # => nil x&.foo < 42 # => nil x&.foo == 42 # => false ### This is where the precedence of &. is higher x&.foo || 42 # => 42 x&.foo ? 1 : 2 # => 2
If you substitute x&.foo
for x && x.foo
in the above, they all evaluate the same except for one:
x && x.foo == 42 # => nil
This makes sense, since ==
is a method and not an operator.
Updated by marcandre (Marc-Andre Lafortune) almost 9 years ago
Matz, did you get a chance to think about the "precedence" level of &.
? It would be harder to change after christmas...
Updated by shugo (Shugo Maeda) almost 9 years ago
Marc-Andre Lafortune wrote:
Matz, did you get a chance to think about the "precedence" level of
&.
? It would be harder to change after christmas...
Is it really hard to change after the release of Ruby 2.3?
IMHO, the current behavior of x&.foo.bar
is useless, so users have to use x&.foo&.bar
instead.
Even if the behavior of x&.foo.bar
is changed as you expect, x&.foo&.bar
still works, and users
can switch from x&.foo&.bar
to x&.foo.bar
gradually.
Please tell me if I miss anything.
I'm not against the proposal itself, but there's no enough time left....
FYI, the behavior of Groovy's ?.
seems to be the same as Ruby's &.
:
$ groovy -e 'x = null; print(x?.foo.bar)'
Caught: java.lang.NullPointerException: Cannot get property 'bar' on null object
java.lang.NullPointerException: Cannot get property 'bar' on null object
at script_from_command_line.run(script_from_command_line:1)
Please tell me if someone knows the behavior in C#.
Updated by marcandre (Marc-Andre Lafortune) almost 9 years ago
Shugo Maeda wrote:
Is it really hard to change after the release of Ruby 2.3?
True, it could be changed afterwards, although it is far from ideal.
Please tell me if someone knows the behavior in C#.
I don't know, but I can say that both CoffeeScript and Swift do the correct thing with respect to x&.foo.bar
.
Updated by mame (Yusuke Endoh) almost 9 years ago
Shugo Maeda wrote:
Is it really hard to change after the release of Ruby 2.3?
As Marc-Andre stated in https://bugs.ruby-lang.org/issues/11816#note-7, it will bring incompatibility. If we may change the spec in future, I think we should explicitly state the possibility in doc and release message. Also, it might be a good idea to mark the feature as "experimental".
Another idea for the proposal: how about propagating &.
as long as explicit method chain (that uses .
literally) continues?
x&.foo * 42 == (x&.foo) * 42
x&.foo.*(42) == (x&.foo&.*(42)) # strictly not equivalent in the case where #foo returns nil
x&.foo[42]
is still ambiguous. `(x&.foo)[42]' looks simple and consistent to me.
--
Yusuke Endoh mame@ruby-lang.org
Updated by shugo (Shugo Maeda) almost 9 years ago
Yusuke Endoh wrote:
Is it really hard to change after the release of Ruby 2.3?
As Marc-Andre stated in https://bugs.ruby-lang.org/issues/11816#note-7, it will bring incompatibility. If we may change the spec in future, I think we should explicitly state the possibility in doc and release message. Also, it might be a good idea to mark the feature as "experimental".
It would be better if Matz would like to change the behavior in future.
Another idea for the proposal: how about propagating
&.
as long as explicit method chain (that uses.
literally) continues?x&.foo * 42 == (x&.foo) * 42 x&.foo.*(42) == (x&.foo&.*(42)) # strictly not equivalent in the case where #foo returns nil
In my understanding, rather than (x&.foo&.*(42))
, x&.foo.*(42)
is equivalent to x && x.foo.*(42)
except when x
is false
.
So, the phrase "propagating &.
" is confusing, isn't it?
Anyway, your proposal sounds reasonable because x&.foo * 42
is parsed as (x && x.foo) * 42
rather than x && (x.foo * 42)
in my brain.
However, the behavior like (x && x.foo) * 42
seems to be useless for the same reason as the current behavior of x&.foo.bar
, so there is a trade-off.
Updated by shugo (Shugo Maeda) almost 9 years ago
Shugo Maeda wrote:
Anyway, your proposal sounds reasonable because
x&.foo * 42
is parsed as(x && x.foo) * 42
rather thanx && (x.foo * 42)
in my brain.
Just for clarification, x&.foo.bar
as (x && x.foo).bar
looks more natural to me than x && x.foo.bar
.
However, the behavior like
(x && x.foo) * 42
seems to be useless for the same reason as the current behavior ofx&.foo.bar
, so there is a trade-off.
So there is also a trade-off between the current behavior and Endoh-san's proposal.
Updated by naruse (Yui NARUSE) almost 9 years ago
I understand this as the idea returned value of safe navigation operator behaves like NaN while the method chain.
It sounds interesting and feels bit useful but I weakly object it.
Anyway using && in &.'s explanation should be harmful especially this serious topic because the behavior is different in handling false.
Updated by marcandre (Marc-Andre Lafortune) almost 9 years ago
Matz, is it possible to have a decision before the official release, so we at least know what the future might bring and can introduce it properly?
I still hope my "whichever is the most useful" proposal is accepted. We look at it right now with a mental magnifying glass, but I think that it would become second nature in practice, as I feel it would usually be what the rubyist wants. To me, it feels similar to super
which passes argument and block; not necessarily intuitive when you think about it, but if you don't think about it, it does usually what you want to do.
Thanks
Updated by marcandre (Marc-Andre Lafortune) almost 9 years ago
Dear Matz,
Any update on the precedence of &.
?
Updated by matz (Yukihiro Matsumoto) almost 9 years ago
Marc,
I don't feel it's right to ignore nils in calls after &.
Nil-ignorance should be explicit, I think.
Matz.
Updated by marcandre (Marc-Andre Lafortune) almost 9 years ago
So if I understand correctly, you like the current behavior, although it is not useful in any circumstance I can think of?
e_error_bytes&.dup.force_encoding(...) #=> NoMethodError if error_bytes is nil
So there are basically no circumstances where one would write foo&.bar
and follow it with .
, <
, ==
, etc
Even Swift allows x&.foo.bar
to be something meaningful.
Updated by seanlinsley (Sean Linsley) over 8 years ago
I prefer Marc's proposal here: https://bugs.ruby-lang.org/issues/11816#note-6. I think that's the much more natural than the existing behavior. I found this ticket after being surprised by the behavior, as I attempted to update my project to use &.
. Most of the places I would want to use &.
currently have an if condition wrapping them, because the first couple objects in the chain may not exist. With the feature as it is currently, it's pretty much useless, because this is unacceptably hard to read:
book&.authors&.last&.first_name
Updated by shyouhei (Shyouhei Urabe) over 8 years ago
I was not active when this topic was hot, so to know the current status I asked this on this month's developer meeting. The bottom line is there was no concrete answer for this issue yet.
At first "propagating &. as long as explicit method chain continues" seemed to be the right way. But we found there still are edge cases that should be clearly described before adopting that way. Consider:
a&.b.c += d
here, if it was a.b.c += d
, it calls two methods a.b.c
and a.b.c=
. Then what should happen for &.
?
Updated by marcandre (Marc-Andre Lafortune) over 8 years ago
I feel that a&.b.c += d
is still an assignment, so should be treated with the same precedence.
a = nil
a&.b.c += d # => nil
a&.b.c ||= d # => nil
a&.b.c &&= d # => nil
# etc...
I think this is the most helpful solution.
Moreover, we already have (in 2.3.0)
a&.b += d # => nil
a&.b ||= d # => nil
a&.b &&= d # => nil
Given this, I feel the resolution I'm giving is also the most intuitive.
Just to be sure, with "propagating &. as long as explicit method chain continues", we consider that x&.foo * 42
returns nil
, as calling *
is equivalent to .*
, right?
Updated by nobu (Nobuyoshi Nakada) over 8 years ago
Marc-Andre Lafortune wrote:
Just to be sure, with "propagating &. as long as explicit method chain continues", we consider that
x&.foo * 42
returnsnil
, as calling*
is equivalent to.*
, right?
I think so.
Updated by sawa (Tsuyoshi Sawada) over 8 years ago
What about allowing parentheses after the safe navigation operator like this:
foo&.(bar.baz)
which should behave in the way proposed.
The edge cases:
Shyouhei Urabe wrote:
a&.b.c += d
with the two possibilities involved can be distinguished as:
a&.(b.c) += d # non-useful one a&.(b.c += d)
Does this conflict with the current syntax?
Or, if bar
above is confusing (to human) with a local variable, then another option may be:
foo&(.bar.baz)
Updated by nobu (Nobuyoshi Nakada) over 8 years ago
- Description updated (diff)
Tsuyoshi Sawada wrote:
Does this conflict with the current syntax?
Yes.
proc{|x|p x}&.(1) #=> 1
Or, if
bar
above is confusing (to human) with a local variable, then another option may be:foo&(.bar.baz)
Is &(.
a single token?
Updated by joanbm (Joan Blackmoore) over 8 years ago
I can understand the intention behind the proposal, but would like to express a (strong) disagreement with it.
The thing is that it would break language's consistency and is in contradiction to declared behavior.
"safe navigation operator" serves, by accepted feature request #11537, as an alternate way of method invocation. The only difference to standard method sending is ignoring passed argument if target object is nil. This means, expression may result either in a value of expected type or nil and programmer need count with this eventuality.
This bug report, or rather feature request, asks for change of existing role of &. as a "safe" methods chain navigator and make it later/last resort objects combinator.
Lowering operator's precedence would result in much harder to understand code, in figuring out at which moment it would be applied and what would be its argument. In comparison to unambiguous situation, when it has the highest implicit precedence, like the standard dot method call.
Setting an explicit precedence by parens would remedy the situation but, as already mentioned by Nobu, it would clash with existing .() syntax, a very unfortunate design decision introduced into the language as a parse-time way of writing Proc/Method#call . Confusing with standard method arguments passing, enforcing eval precedence, optional dot at operator methods… Unless widely used, its removal in Ruby 3.0 would be highly welcomed.
I'd suggest keep the existing behavior or possibly extend with #respond_to?
method check and get rid of exception to the rule when applied to NillClass instance, like nil&.nil?
resulting in true
.
It may also be taken into the consideration a little inconsistency when followed by operator method:
optional dot omitted
40+2 # 42
40&.+2 # 42
vs
40.+2 # 42
40&..+2 # syntax error
Updated by phluid61 (Matthew Kerwin) over 8 years ago
I don't necessarily disagree with the rest of Joan's post, but for this point:
Joan Blackmoore wrote:
It may also be taken into the consideration a little inconsistency when followed by operator method:
optional dot omitted
40+2 # 42
40&.+2 # 42
vs
40.+2 # 42
40&..+2 # syntax error
That's perhaps not the right consistency to be searching for; the &.
operator is a dressed up .
operator. There's no equivalent sugar-coated safe operator syntax.
40 + 2 # "unsafe"
#??? # "safe"
40.+2 # "unsafe"
40&.+2 # "safe"
Updated by joanbm (Joan Blackmoore) over 8 years ago
Thought about it again and would agree with the last paragraph. Direct substitution is not appropriate here, despite it sounds logical.
The &. operator is a strange beast as other general rules also won't apply, like (optional) preceding dot, ie. object.&.hash is also a syntax error.
Btw. by playing with different &. contained expressions, I've discovered a possible bug:
This runs ok
a = nil
a&.foo &&= false # nil
Following however freezes VM and need to be SIGKILLed
nil&.foo &&= false
Even at bytecode compilation
RubyVM::InstructionSequence.compile('nil&.foo ||= 42')
tail of strace output
lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/usr/lib64", {st_mode=S_IFDIR|0755, st_size=139264, ...}) = 0
lstat("/usr/lib64/ruby", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/usr/lib64/ruby/2.3.0", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/usr/lib64/ruby/2.3.0/unicode_normalize.rb", {st_mode=S_IFREG|0644, st_size=3265, ...}) = 0
Tested with
ruby -v
ruby 2.3.0p75 (2016-04-07 revision 54505) [x86_64-linux]
command used
strace ruby --disable=gems,rubyopt -e 'nil&.foo &&= false'
Updated by nobu (Nobuyoshi Nakada) over 8 years ago
- Status changed from Open to Closed
Applied in changeset r54628.
compile.c: fix dangling link
- compile.c (iseq_peephole_optimize): should not replace the
current target INSN, not to follow the replaced dangling link in
the caller. [ruby-core:74993] [Bug #11816]
Updated by nobu (Nobuyoshi Nakada) over 8 years ago
- Status changed from Closed to Open
- Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN to 2.1: DONTNEED, 2.2: DONTNEED, 2.3: UNKNOWN
Updated by naruse (Yui NARUSE) over 8 years ago
- Precedes Bug #12296: [Backport] fix dangling link added
Updated by naruse (Yui NARUSE) over 8 years ago
- Status changed from Open to Closed
Applied in changeset ruby_2_3|r54635.
merge revision(s) 54628: [Backport #12296]
* compile.c (iseq_peephole_optimize): should not replace the
current target INSN, not to follow the replaced dangling link in
the caller. [ruby-core:74993] [Bug #11816]
Updated by naruse (Yui NARUSE) over 8 years ago
- Status changed from Closed to Assigned
Updated by joanbm (Joan Blackmoore) over 8 years ago
Oh, thanks for the fix. That was really fast.
Updated by marcandre (Marc-Andre Lafortune) almost 7 years ago
- Tracker changed from Bug to Feature
- ruby -v deleted (
preview 2) - Backport deleted (
2.1: DONTNEED, 2.2: DONTNEED, 2.3: UNKNOWN)
Matz, did you get a chance to consider the precedence of &.
?
As a reminder, there's currently no real use for foo&.bar.baz
or similar. We are forced to write foo&.bar&.baz
even though this could introduce unwanted errors, e.g. if bar
was erroneously returning nil
.
I still believe that my proposal https://bugs.ruby-lang.org/issues/11816#note-6 is the right one.
Thanks for the consideration.
Updated by znz (Kazuhiro NISHIYAMA) almost 7 years ago
marcandre (Marc-Andre Lafortune) wrote:
As a reminder, there's currently no real use for
foo&.bar.baz
or similar. We are forced to writefoo&.bar&.baz
even though this could introduce unwanted errors, e.g. ifbar
was erroneously returningnil
.
I found real use case in 【アンチパターン】全部nil(null)かもしれない症候群.
if friend&.message.blank?
Updated by sawa (Tsuyoshi Sawada) almost 7 years ago
znz (Kazuhiro NISHIYAMA) wrote:
marcandre (Marc-Andre Lafortune) wrote:
As a reminder, there's currently no real use for
foo&.bar.baz
or similar. We are forced to writefoo&.bar&.baz
even though this could introduce unwanted errors, e.g. ifbar
was erroneously returningnil
.I found real use case in 【アンチパターン】全部nil(null)かもしれない症候群.
if friend&.message.blank?
I don't think that is a real use case. The linked article mentions coding style. It suggests not to use the safe navigation operator without thinking or when it is not necessary. The feature proposed in this thread actually makes these things happen automatically, and will remove the necessity of such consideration. It is actually consistent with the current proposal.
Updated by phluid61 (Matthew Kerwin) almost 7 years ago
sawa (Tsuyoshi Sawada) wrote:
I don't think that is a real use case. The linked article mentions coding style. It suggests not to use the safe navigation operator without thinking or when it is not necessary. The feature proposed in this thread actually make these things happen automatically, and will remove the necessity of such consideration. It is actually consistent with the current proposal.
I think .nil?
, .inspect
, .tap
could all be legitimately used on a value or nil.
It makes sense to me that &.
is just a safe version of .
, not some sort of "infectious nil" operation that propagates until an arbitrary precedence limit is hit. If there was a way to explicitly signal the end of the "infectious nil" I'd probably find it useful (for example, a nicer version of x&.instance_eval{foo.bar}
), but then I don't think that's &.
; it's something new.
Updated by marcandre (Marc-Andre Lafortune) almost 7 years ago
phluid61 (Matthew Kerwin) wrote:
If there was a way to explicitly signal the end of the "infectious nil" I'd probably find it useful
There is one way, and it is the same as with all the cases where the precedence doesn't go the way you want it: parentheses.
(foo || bar) && baz
(friend&.message).blank?
The whole point of precedence is to allow writing things simply and without parentheses most of the time.
Matz: was this discussed at the developers' meeting?
Updated by phluid61 (Matthew Kerwin) almost 7 years ago
marcandre (Marc-Andre Lafortune) wrote:
phluid61 (Matthew Kerwin) wrote:
If there was a way to explicitly signal the end of the "infectious nil" I'd probably find it useful
There is one way, and it is the same as with all the cases where the precedence doesn't go the way you want it: parentheses.
(foo || bar) && baz (friend&.message).blank?
The whole point of precedence is to allow writing things simply and without parentheses most of the time.
Hmm, that's almost right, but in my mind it still doesn't quite fit. I think the problem I have is the expectation that modifying one message dispatch (.
→ &.
) shouldn't affect subsequent messages. If you want to affect the dispatch of a group of messages you should use a scoping construct, and operator precedence (even with parens) isn't scope. That's why foo&.instance_eval{bar.baz}
feels right, even if it's ugly.
Perhaps I am alone in seeing &.
as an armoured .
, not an executing &&
. (Despite the fact that false&.! == true
)
Updated by marcandre (Marc-Andre Lafortune) over 4 years ago
FWIW, EcmaScript 2020 introduced a similar operator, see https://github.com/tc39/proposal-optional-chaining/
. The precedence is between []
and *
, higher than I propose in https://bugs.ruby-lang.org/issues/11816#note-6 but at least low enough to be actually useful.
Matz, any chance we'll have a more useful operator precedence in Ruby 3?
Updated by Dan0042 (Daniel DeLorme) over 4 years ago
The &.
operator has pretty well-defined semantics by now and changing it may break some existing code. Maybe a possible alternative would be a similar but separate operator like &&.
/ and.
which would then intuitively have the same precedence as &&
/ and
.
x and .foo.bar
reads quite well to me, better than x&.foo.bar
Updated by marcandre (Marc-Andre Lafortune) over 4 years ago
The &. operator has pretty well-defined semantics by now and changing it may break some existing code.
Any change may break some existing code. Given the fact that foo&.bar.baz
have basically no use whatsover, it is difficult to see what functioning code could be broken though. Let's also think of how many potential bugs we are creating by forcing Rubyists to write foo&.bar&.baz
when that's not what they mean.
Updated by zverok (Victor Shepelev) over 4 years ago
Given the fact that
foo&.bar.baz
have basically no use whatsover, it is difficult to see what functioning code could be broken though
I can imagine some!
- Not from production, but it does not seem too obscure:
foo&.bar.nil?
would betrue
in the current implementation, andnil
in "skip-the-rest" implementation. - A bit more obscure, but this one is from real code:
find.some.array&.first.tap { |val| log.debug "nothing found" if val.nil? }
- ... or
find_key_vaue_pair(hash)&.last.then { |val| val.nil? ? default : val }
(not just||
, becausefalse
is acceptable value)
Obviously, all can be rewritten another way -- but obviously, all will be currently broken by the change proposed.