Project

General

Profile

Feature #15973

Make it so Kernel#lambda always return a lambda

Added by alanwu (Alan Wu) 19 days ago. Updated 4 days ago.

Status:
Open
Priority:
Normal
Assignee:
-
Target version:
-
[ruby-core:93482]

Description

When Kernel#lambda receives a Proc that is not a lambda,
it returns it without modification. l propose changing Kernel#lambda
so it always returns a lambda.

Calling a method called lambda and having it effective do nothing was
not very intuitive.

https://github.com/ruby/ruby/pull/2262

Judging from marcandre's investigation here: https://bugs.ruby-lang.org/issues/15620#note-1
changing the behavior should not cause much breakage, if any.

This also happens to fix [Bug #15620]


Related issues

Related to Ruby master - Feature #12957: A more OO way to create lambda ProcsFeedbackActions
Related to Ruby master - Feature #7314: Convert Proc to Lambda doesn't work in MRIAssignedActions
Related to Ruby master - Feature #9777: Feature Proposal: Proc#to_lambdaFeedbackActions
Related to Ruby master - Feature #8693: lambda invoked by yield acts as a proc with respect to returnClosed07/26/2013Actions
Related to Ruby master - Bug #16004: Kernel#lambda captured with Kernel#method doesn't create lambdasOpenActions

History

Updated by matz (Yukihiro Matsumoto) 19 days ago

I agree. Even though we have to investigate how big the consequence of the change first.

Matz.

#2

Updated by alanwu (Alan Wu) 19 days ago

  • Subject changed from Make Kernel#lambda always return lambda to Make it so Kernel#lambda always return a lambda

Updated by Eregon (Benoit Daloze) 19 days ago

I'm not sure changing this is good, because it can be very surprising for the code to change semantics dynamically.
Also, should proc(&lambda) make a non-lambda Proc then? It would be inconsistent if not.

As I said in https://bugs.ruby-lang.org/issues/15620#note-2, the current rule is AFAIK only change the semantics if a block is directly given, because the programmer means to use those semantics then.
If not, don't change the semantics as it would break the user (the Proc's code)'s intention.
The only exception to this is define_method with a pre-existing Proc, which is very rare, and understandably needed because methods should check arguments strictly. (maybe we don't need that exception)

I believe #15620 should be fixed on its own, it's a bug of the optimization.

Here is an example (a bit contrived, I'm tired):

def foo(arg)
  early_check = Proc.new {
    # everything here assumes it will always be proc/non-lambda semantics
    return :early_return if arg > 3
  }

  # Some way for some external code to access `early_check`
  early_check = yield early_check

  early_check.call

  :method_return_value
end

p foo(4) { |pr| lambda(&pr) }

With current semantics, it returns :early_return.
With the proposed change, it returns :method_return_value.
That's very surprising, isn't it? The code clearly means it wants a non-local return to exit the method,
and yet somehow it was transformed into a local lambda return!

I think such a surprising transformation of user code should happen as little as possible, so I'm against this proposal.

Maybe we should simply forbid calling proc/lambda without a literal block (i.e., with an explicit Proc like lambda(&pr)), since it doesn't do anything useful ?
That would make more sense to me, and be in line with ko1 (Koichi Sasada)'s recent changes to disallow things like Proc.new, etc without a block.

#4

Updated by shyouhei (Shyouhei Urabe) 19 days ago

  • Related to Feature #12957: A more OO way to create lambda Procs added
#5

Updated by shyouhei (Shyouhei Urabe) 19 days ago

  • Related to Feature #7314: Convert Proc to Lambda doesn't work in MRI added

Updated by shyouhei (Shyouhei Urabe) 19 days ago

No. I'm against it. We have discussed this before multiple times. See the issues I linked just now.

Updated by alanwu (Alan Wu) 14 days ago

The 2.4 spec is a bit problematic since it makes it impossible to forward a
block to Kernel#lambda with a block pass. In the rest of the language
forwarding a block has no effect on semantics compared to passing one
literally. The idea of literal blocks being different from non-literal ones
exists only in the case of Kernel#lambda.

A concept at odds with the rest of the language specific to one single method
is sure to surprise. Special one-off concepts make the language harder to learn
and add complications to implementaitons.

As others have pointed out, it's already possible to transform a proc into a
lambda-proc via define_method. My proposal isn't adding anything new with
regards to messing with the perscribed usage of code within blocks.

Transforming a proc into a lambda-proc is certainly a sharp tool. For me it
falls in the same category as instance_variable_set and const_set. However,
I think the situation in which it ends up being surprinsg is very rare.
return within a Proc.new do ... comes up rarely as is.

Is "someone might misuse this in specific sutations" a good reason to keep an
ad-hoc concept in the language? For me, no.

Updated by alanwu (Alan Wu) 14 days ago

I would also like to note that if we revert back to the behavior in 2.4, code that relied on lambda without block will not have an easy upgrade path.

def make_a_lambda
  lambda
end

is not equivalent to

def make_a_lambda(&block)
  lambda(&block)
end

under 2.4 spec.

Updated by shyouhei (Shyouhei Urabe) 13 days ago

"It's already broken, why not break it more" is not what I can follow.

Can I ask you why you need this feature? If this not more than a matter of consistency, I would like to second Eregon (Benoit Daloze)'s proposal: lambdas without literal blocks to be prohibited at all.

Updated by alanwu (Alan Wu) 12 days ago

Can I ask you why you need this feature?

I want to be able to forward a block to Kernel#lambda.

Kernel#lambda is in a weird spot. Even though it's a method, making it behave like one has the unfortunate side-effect of allowing proc transformation.
On the other hand, restricting it to just literal blocks is fairly magical and makes it just another way to do stabby lambda.
It seems like there is no perfect solution here.
I don't feel too strongly about my proposal. Banning block-pass to Kernel#lambda sounds good to me too if others are not comfortable with making proc-to-lambda transformation more accessible.

Updated by Eregon (Benoit Daloze) 12 days ago

alanwu (Alan Wu) wrote:

Kernel#lambda is in a weird spot. Even though it's a method, making it behave like one has the unfortunate side-effect of allowing proc transformation.

Yes, the semantics of Kernel#lambda have always been a bit weird because no method should be allowed to convert a Proc to a lambda and inversely.
And visually, lambda { ... } receives a block, which is a non-lambda Proc.

My point of view is lambda is a relic of the past, and we should use -> { ... } instead, which has clear semantics and obviously you can't pass a pre-existing block to it.
Yes, I think we should raise lambda(&existing_block) as that can't behave intuitively (it seems either useless or confusing).

Updated by akr (Akira Tanaka) 11 days ago

I'm against it.

Changing lambda <-> proc can violate the intent of programmers who write a block.

It seems good to raise an exception at lambda(&b) where b is (non-lambda) proc.

Updated by matz (Yukihiro Matsumoto) 11 days ago

In my opinion, lambda should return lambda object as a principle. We need to keep the discussion.
Maybe we can generate a delegating lambda object in this case.

Matz.

Updated by knu (Akinori MUSHA) 11 days ago

I don't think the lambda flag should be altered after the creation of a proc, because it's all up to the writer of a block how return/break etc. in it should work.

What about just deprecate lambda in the long run in favor of the lambda literal ->() {}?

Updated by marcandre (Marc-Andre Lafortune) 10 days ago

knu (Akinori MUSHA) wrote:

I don't think the lambda flag should be altered after the creation of a proc, because it's all up to the writer of a block how return/break etc. in it should work.

akr wrote similar opinion.

This same opinion would call for deprecation of define_method(&block), which I believe would be a mistake.

I am assuming it is clear that lambda(&block) would never mutate block and return a new object that would be a lambda.

Updated by shyouhei (Shyouhei Urabe) 10 days ago

Suppose we change the spec so that lambda(&nonlambda) generates a lambda.
Suppose we have the following nonlambda proc:

1: foldr = proc do |x, *xs|
2:   foo(x, foldr.(xs)) if x ||! xs.empty?
3: end

A straight-forward foldr implementation, with foo defined elsewhere.
Now let's "convert" it into a lambda:

4: foldr = lambda(&foldr)

This magically breaks the program. The meaning of |x, *xs| changed.
The problem I see is that the broken program would generate a backtrace like this:

Traceback (most recent call last):
        5444: from tmp.rb:2:in `block in <main>'
        5443: from tmp.rb:2:in `block in <main>'
        5442: from tmp.rb:2:in `block in <main>'
        5441: from tmp.rb:2:in `block in <main>'
        5440: from tmp.rb:2:in `block in <main>'
        5439: from tmp.rb:2:in `block in <main>'
        5438: from tmp.rb:2:in `block in <main>'
         ... 5433 levels...
           4: from tmp.rb:2:in `block in <main>'
           3: from tmp.rb:2:in `block in <main>'
           2: from tmp.rb:2:in `block in <main>'
           1: from tmp.rb:2:in `block in <main>'
tmp.rb:2:in `block in <main>': stack level too deep (SystemStackError)

There is no single bit of information that the prolem is caused by that lambda conversion.
An end user has no other way than to blame tmp.rb:2 not tmp.rb:4, because the backtrace says so.

This is really bad. Think of for instance the block was from another library. The library authors suddenly get angry bug reports due to some random other guy turned their proc into lambda. This is nonsense.

Non-lambda to lambda conversion disturbs the boundary point of responsibility. Should not be allowed I believe.

Updated by alanwu (Alan Wu) 10 days ago

The idea to generate a delegating lambda seems to side-step a lot of the issues posted here.
For reference here are some quotes from the log in #15930 (courtesy of ko1):

b = proc {|x| x }
lambda(&b) #=> lambda {|x| b[x] }

b = proc {|x, y, k:1| x }
lambda(&b) #=> lambda {|x, y, k:1| b[x, y, k:k] }

This preserves the return semantics in the original proc. I would imagine it would also add an entry to the call stack.
(side note, I think the default parameter evaluation has to be left to the original proc too since a return could be in there)

What do you folks think about this?

#18

Updated by akr (Akira Tanaka) 10 days ago

  • Related to Feature #9777: Feature Proposal: Proc#to_lambda added
#19

Updated by akr (Akira Tanaka) 10 days ago

  • Related to Feature #8693: lambda invoked by yield acts as a proc with respect to return added

Updated by shyouhei (Shyouhei Urabe) 10 days ago

Yes, I can compromise with delegation scheme, as long as it leaves its own frame in the call stack. An optional method for a proc to prevent such delegation is desirable but can be added later.

Updated by akr (Akira Tanaka) 10 days ago

Eregon (Benoit Daloze) wrote:

With current semantics, it returns :early_return.
With the proposed change, it returns :method_return_value.
That's very surprising, isn't it?

I agree.

The lambda-ness of Proc object affects control flow: the behavior of "return" and "break".

If lambda(&b) changes the lambda-ness of b, two control flow can exist.
I think no programmer want to consider two control flow when implementing one block.

Updated by Eregon (Benoit Daloze) 10 days ago

alanwu (Alan Wu) wrote:

The idea to generate a delegating lambda seems to side-step a lot of the issues posted here.

So then, would it be the same semantics as this?

b = proc {|x, y, k:1| x }
l = lambda { |*args, **kwargs, &block| b.call(*args, **kwargs, &block) }

I think the lambda cannot have the same (in the code) arguments as the proc, otherwise https://bugs.ruby-lang.org/issues/15973#note-16 would fail too.

alanwu (Alan Wu) Can you present use case(s) for turning procs into lambdas?
So far I only see consistency but the delegating lambda is not that consistent with #lambda with a literal block.
Without a good use-case, I think raising on #lambda without a literal block would be the safest and least surprising.

#23

Updated by Eregon (Benoit Daloze) 7 days ago

  • Related to Bug #16004: Kernel#lambda captured with Kernel#method doesn't create lambdas added

Updated by alanwu (Alan Wu) 4 days ago

Can you present use case(s) for turning procs into lambdas?

One use case is reading the parameters of blocks as if they were written for a method definition: https://bugs.ruby-lang.org/issues/9777#note-11
Another one is getting lambda style parameter checking for the incoming block. msgpack-ruby does this through the CAPI: https://github.com/msgpack/msgpack-ruby/blob/b48f9580bfd33be6a86652f10a41bd691a7d0c91/ext/msgpack/unpacker_class.c#L367

I asked for opinions about the delegation scheme because it seems like a good way to go if we must return a lambda. At the time I didn't make up my mind as to whether it would be a good idea to return one.
I now believe that banning everything but the literal block form lambda is overall better. lambda being a method exposes it to many different ways of calling it, some of which don't have obvious semantics.
We can pick a semantic for lambda(&thing), but there will always be some percentage of users that find the behavior surprising. We can minimizes surprises, but we can't eliminate it, as long
as lambda is a method.

To illustrate, here are two more ways to call into Kernel#lambda that are hard to define good semantics for: super (zsuper) and method(:lambda).call {}. Right now zsuper creates a lambda while
method(:lambda).call {} doesn't. I'm not sure what these should do and different people probably have different ideas.

I think lambda was supposed to be a psudo-keyword but implementing it as a method is unfortunately exposing it to a whole array of usage that were not considered.
If we can promote it to a psudo-keyword, that would be best. Failing that, banning lambda(&thing) at least removes one source of surprise with clear messaging.

Also available in: Atom PDF