Project

General

Profile

Actions

Bug #17429

open

Prohibit include/prepend in refinement modules

Added by shugo (Shugo Maeda) 9 months ago. Updated 11 days ago.

Status:
Open
Priority:
Normal
Target version:
-
[ruby-core:101639]

Description

include/prepend in refinement modules has implementation difficulties such as #17007 and #17379, and tends to be misleading like #17374.
How about to prohibit it in future versions?

Method copy like #17380 may be more convenient, but it's confusing to use names include and prepend because semantics is different from the original ones.


Related issues

Related to Ruby master - Bug #17007: SystemStackError when using super inside Module included and lexically inside refinementAssignedshugo (Shugo Maeda)Actions
Related to Ruby master - Bug #17374: Refined methods aren't visible from a refinement's moduleRejectedshugo (Shugo Maeda)Actions
Related to Ruby master - Bug #17379: Refinement with modules redefinition bugOpenko1 (Koichi Sasada)Actions
Related to Ruby master - Bug #18021: Mixins in Refinements: possibly multiple bugs, workarounds are awkwardOpenActions

Updated by matz (Yukihiro Matsumoto) 9 months ago

I basically agree. Combination of refinement and include/prepend only cause confusion.

Matz.

Updated by Eregon (Benoit Daloze) 9 months ago

+1 from me!

I think a new Module method to copy all methods to another Module could be useful.
Something like A.copy_methods(B).
It seems there is no need to copy constants, because the constant scope of "copied" methods would still be A (and lexical parents).
Actually the method would only do a shallow copy of each method, i.e., still use the same bytecode, etc, so maybe another name than copy_methods would be clearer.

The docs of Module#append_features make it sounds like #append_features would do that, but it doesn't.
Actually, #append_features adds the given module in the ancestors chain (include = append_features + included).
Would be a good occasion to clarify the docs of #append_features.

Updated by marcandre (Marc-Andre Lafortune) 9 months ago

Is there a proposal to import modules in refinements?

Something like this?

module Code
  # ...
end

refine Object, import: Code do
  # extra methods
end

I still think that include and prepend within the refine block could have that function.

I think that having a nice way to implement methods that can be used with include or using would help adoption of refinements for gems.

Updated by Dan0042 (Daniel DeLorme) 9 months ago

I agree the current situation needs to be fixed, and prohibiting include/prepend is the simplest way. But I also think there has to be a way to achieve what Marc-Andre was trying in #17374.

marcandre (Marc-Andre Lafortune) wrote in #note-3:

refine Object, import: Code do
  # extra methods
end

That looks pretty good I think.

I still think that include and prepend within the refine block could have that function.

But there's not much benefit to that is there? Having a different name such as import feels cleaner. Using include or prepend within a refine block could result in a warning/error along the lines of "include is not supported in refinements but you can use the almost-equivalent import argument."

Actions #6

Updated by jeremyevans0 (Jeremy Evans) 9 months ago

  • Related to Bug #17007: SystemStackError when using super inside Module included and lexically inside refinement added
Actions #7

Updated by jeremyevans0 (Jeremy Evans) 9 months ago

  • Related to Bug #17374: Refined methods aren't visible from a refinement's module added
Actions #8

Updated by jeremyevans0 (Jeremy Evans) 9 months ago

  • Related to Bug #17379: Refinement with modules redefinition bug added

Updated by matsuda (Akira Matsuda) 9 months ago

Calling include in refinement modules has certin use cases.

Here's an actual example.
https://github.com/tomykaira/rspec-parameterized/blob/v0.4.2/lib/rspec/parameterized/table_syntax.rb#L27-L61

This gem uses Module#include to avoid code repetition, which to me looks quite natural and basic usage of Module.

If we prohibit include in refinement modules, can this code still be written equally simply?

Updated by Eregon (Benoit Daloze) 9 months ago

With https://bugs.ruby-lang.org/issues/17429#note-3 it would.
I think it makes sense adding such functionality at the same time as no longer allowing include for refinement modules.

Updated by shugo (Shugo Maeda) 9 months ago

marcandre (Marc-Andre Lafortune) wrote in #note-3:

Is there a proposal to import modules in refinements?

Something like this?

module Code
  # ...
end

refine Object, import: Code do
  # extra methods
end

I prefer the following way, but I'm not sure about the name import.

refine Object do
   import Code
end

The behavior is similar to Module#mix proposed by Matz before.

Updated by Dan0042 (Daniel DeLorme) 9 months ago

shugo (Shugo Maeda) wrote in #note-11:

The behavior is similar to Module#mix proposed by Matz before.

Visually it's a more pleasing API than the import keyword, but would it be available in any module or just refinements?
If any module, we'd now have three mixin mechanisms: include, prepend, import. IMO that's overly complex.
If just refinements, it feels inconsistent. IMO we'll have people asking why they can't use import in classes and modules.
As a keyword it's clear this is a refinement-only behavior.

Updated by shugo (Shugo Maeda) 8 months ago

Dan0042 (Daniel DeLorme) wrote in #note-12:

shugo (Shugo Maeda) wrote in #note-11:

The behavior is similar to Module#mix proposed by Matz before.

Visually it's a more pleasing API than the import keyword, but would it be available in any module or just refinements?
If any module, we'd now have three mixin mechanisms: include, prepend, import. IMO that's overly complex.
If just refinements, it feels inconsistent. IMO we'll have people asking why they can't use import in classes and modules.
As a keyword it's clear this is a refinement-only behavior.

It's enough to changing the class of a module created by refine to the following subclass of Module, isn't it?

class Refinement < Module
  [:include, :prepend].each do |name|
    define_method(name) do |*args|
      warn("#{name} in a refinement is deprecated; use mix instead", uplevel: 1, category: :deprecated)
      super(*args)
    end
  end

  def mix(*args)
    # ...
  end
end

Updated by Eregon (Benoit Daloze) 8 months ago

Having a Module subclass for Refinements seems nice and useful :+1:

Updated by marcandre (Marc-Andre Lafortune) 8 months ago

I like shugo (Shugo Maeda)'s approach too.

For anyone interested, I released the refine_export gem that makes @jeremyevans' nice hack easy to use:
https://github.com/marcandre/refine_export#usage

Updated by shugo (Shugo Maeda) 8 months ago

  • Assignee set to matz (Yukihiro Matsumoto)

Eregon (Benoit Daloze) marcandre (Marc-Andre Lafortune) Thanks for your feedback.

The remaining issue is the name of the new method.
I came up with the following options:

  1. Refinement#import
  2. Refinement#mix
  3. Refinement#include (different behavior from Module#include)

matz (Yukihiro Matsumoto) Which do you like, or do you have another option in mind?

Updated by ko1 (Koichi Sasada) 8 months ago

I like Module#mix for all modules (not only for refimement).

Updated by Eregon (Benoit Daloze) 8 months ago

ko1 (Koichi Sasada) wrote in #note-17:

I like Module#mix for all modules (not only for refimement).

I think they need different semantics.
For refinements, we will need to do a deep copy of the method, or at least of the inline caches, so that the copied methods see the other refined methods of the refinement module.
That's quite expensive in footprint, but it probably makes sense for this use case with refinements.

For a general Module#mix, I don't think that is needed. Also what's the advantage of a general Module#mix over include/prepend?

Actions #19

Updated by Eregon (Benoit Daloze) 3 months ago

  • Related to Bug #18021: Mixins in Refinements: possibly multiple bugs, workarounds are awkward added

Updated by Eregon (Benoit Daloze) 3 months ago

One more issue reported due these confusing semantics of include/prepend inside refine: #18021.

I suggest we warn in 3.1, raise in 3.2.
And I suggest to add Refinement#import, because:

  • mix seems to imply other things, and if it's ever added to Module with different semantics we will just get more confusion.
  • include seems likely to cause confusion because the behavior would not be Module#include-like. It is also potentially backward-incompatible, raises the question about what would happen for prepend and consistency. And finnaly include would not longer mean "define higher in the ancestor" for this context, i.e., defining the same method in the refine block would replace, not just override).

Updated by Eregon (Benoit Daloze) 3 months ago

I'll mention it here as it may be helpful.
If you want to define the same method in multiple refinements before this is fixed, the current workaround is to use class_eval/module_eval inside refine and have those shared methods in a String.
Not so pretty, but it works and it's simple.

Updated by shugo (Shugo Maeda) 2 months ago

Eregon (Benoit Daloze) wrote in #note-20:

One more issue reported due these confusing semantics of include/prepend inside refine: #18021.

I suggest we warn in 3.1, raise in 3.2.
And I suggest to add Refinement#import, because:

  • mix seems to imply other things, and if it's ever added to Module with different semantics we will just get more confusion.
  • include seems likely to cause confusion because the behavior would not be Module#include-like. It is also potentially backward-incompatible, raises the question about what would happen for prepend and consistency. And finnaly include would not longer mean "define higher in the ancestor" for this context, i.e., defining the same method in the refine block would replace, not just override).

I've implemented Refinement#import in https://github.com/shugo/ruby/pull/3

In the current implementation, the module in cref is replaced with the refinement like Module#dup, so constants in the imported module are not accessible from the copied methods.
It would be possible to support constant access with a hack on cref, but I'm afraid it may bring new problems.

Maybe Refinement#import_methods is a better name if we keep the current behavior.

Updated by Eregon (Benoit Daloze) 2 months ago

shugo (Shugo Maeda) wrote in #note-22:

In the current implementation, the module in cref is replaced with the refinement like Module#dup, so constants in the imported module are not accessible from the copied methods.

Could you show an example that would not work due to that?
Methods from the imported module should be able to access constants from the imported module, otherwise I think it is very surprising.
They should not be able to access constants from the refinement module, that's fine they were declared in the imported module.

Updated by shugo (Shugo Maeda) 2 months ago

Eregon (Benoit Daloze) wrote in #note-23:

shugo (Shugo Maeda) wrote in #note-22:

In the current implementation, the module in cref is replaced with the refinement like Module#dup, so constants in the imported module are not accessible from the copied methods.

Could you show an example that would not work due to that?
Methods from the imported module should be able to access constants from the imported module, otherwise I think it is very surprising.
They should not be able to access constants from the refinement module, that's fine they were declared in the imported module.

For me, it's surprising if the imported methods cannot access constants of the refinement.
However, constant assignments in refine block define constants not in the refinement but in the outer scope, so it may not be a problem actually.

module Ext
  refine Object do
    X = 1 # defines Ext::X
    const_set(:Y, 2) # defines #<refinement:Object@Extension>::X
  end
end

Is it enough that the imported methods can access only constants in the original context?

Updated by Eregon (Benoit Daloze) 2 months ago

Yes, I think that's completely fine.

In code, this should work:

module Shared
  A = 1
  def foo
    A
  end
end

refine SomeClass do
  import Shared
end

SomeClass.new.foo # => 1

And this should not:

module Shared
  def foo
    A
  end
end

refine SomeClass do
  self::A = 1
  import Shared
end

SomeClass.new.foo # => NameError

That would be the equivalent of dynamic rebinding or so, I think nobody expects that, the constant scope has always been lexical (+ ancestors of the first enclosing module).

Could you add tests (or better, specs under spec/ruby) for that?
Then I think it should be good to go.

Updated by Eregon (Benoit Daloze) 2 months ago

In other words, Refinement#import copies methods but does not attempt to change anything lexical, except which refinements are applied to these methods.

Updated by shugo (Shugo Maeda) 18 days ago

In the current implementation:

  • Refinement#import raises an ArgumentError if the specified module has methods written in C. Should it import C methods without refinements activation?
  • Only methods defined directly in the specified module are imported. Importing ancestors' methods may be confusing because Refinement#import doesn't work with super.

Updated by mame (Yusuke Endoh) 12 days ago

In today's dev meeting, matz accepted the concept, but wanted to take some time to consider the name import.

Updated by mame (Yusuke Endoh) 12 days ago

BTW, the change seems to add a top-level new constant ::Refinement. I'm not against the addition, but unsure about the impact. Is it okay?

Updated by shugo (Shugo Maeda) 11 days ago

mame (Yusuke Endoh) wrote in #note-30:

BTW, the change seems to add a top-level new constant ::Refinement. I'm not against the addition, but unsure about the impact. Is it okay?

I found a gem named refinement....

https://github.com/square/refinement/blob/master/lib/refinement.rb

Updated by shugo (Shugo Maeda) 11 days ago

shugo (Shugo Maeda) wrote in #note-31:

mame (Yusuke Endoh) wrote in #note-30:

BTW, the change seems to add a top-level new constant ::Refinement. I'm not against the addition, but unsure about the impact. Is it okay?

I found a gem named refinement....

https://github.com/square/refinement/blob/master/lib/refinement.rb

I've created an issue on the project: https://github.com/square/refinement/issues/71.

By gem-codesearch, I've found another gem which defines ::Refinement.

However, it's a gem for very old Ruby versions without Refinements, so I believe there's no problem.

Actions

Also available in: Atom PDF