Misc #21154: Document or change Module#autoload?

• Status: Closed
• Assignee: -

[fxn (Xavier Noria)]

The documentation of Module#autoload? says

Returns filename to be loaded if name is registered as autoload in the namespace of mod or one of its ancestors.

Cool, but in the following snippet

autoload :Foo, 'foo'
autoload?(:Foo)

the second line could evaluate to nil, and this does not seem to agree. I just registered an autoload, therefore (according to the documentation) I should get "foo" back in line 2.

I'd like to ask for clarification from the Ruby team:

1. Is the documentation complete? Should that second line always return "foo"?
2. If the answer is no, which is the logic missing in the docs?

Thank you!

[Eregon (Benoit Daloze)]

• Related to Misc #21035: Clarify or redefine Module#autoload? and Module#const_defined? added

[mame (Yusuke Endoh)]

I would like you to learn how to write a bug report.

the second line could evaluate to nil.

Please write a self-contained reproducible example that actually does that, not just say “could”.
Then write down the results you expect to get with that code, and the actual results.

Also, I want you to write the background (what you want to do finally), and why the current behavior is troubling for your purpose.

[mame (Yusuke Endoh)]

Should that second line always return "foo"?

No, not necessarily. The document says "if name is registered as autoload".
And the document of Kernel#autoload says "If const is defined but not as autoload, does nothing."
Thus, there is no guarantee that your second line will always return "foo".

I assume you have https://bugs.ruby-lang.org/issues/21035#note-3 in mind.
IMO, in your case, the situation could be considered equivalent to the “If const is defined but not as autoload, does nothing.” case.
Therefore, I don't think it is surprising that autoload? returns nil.

So should the documentation be updated? I personally think that is not necessary either. Documentation is documentation, not a spec. I think it is more useful to have a simple description of typical behavior and usage. Currently, I feel the document is good enough. I admit people may have different opinions here, though.

I think what you should do is not arguing about consistency with the documentation, but rather briefly explaining your purpose and why the current behavior is troubling against it.

[fxn (Xavier Noria)]

Hi @mame (Yusuke Endoh), thank for your reply.

No, not necessarily. The document says "if name is registered as autoload".
And the document of Kernel#autoload says "If const is defined but not as autoload, does nothing."
Thus, there is no guarantee that your second line will always return "foo".

That is right. However, if we have a file foo.rb

autoload :Foo, 'foo'
p autoload?(:Foo)

That prints nil if you ruby -I. -e 'require "foo"'.

So, the constant is not defined, the autoload is registered, and you get nil. This does not agree with the documentation, do you agree? You may say "circularity!". And I'll say, cool, but undocumented.

I think what you should do is not arguing about consistency with the documentation, but rather briefly explaining your purpose and why the current behavior is troubling against it.

I am not arguing, I believe my tickets cannot be more respectful, formulated as questions, and leaving the door open to an explanation of how things work that is not documented.

You do not always come to ask for questions with a use case in mind. I do use autoload? and this edge case affects me, yes. But what I am trying to accomplish is to know Ruby better.

I understand you don't want to document this (we disagree, but that is your call), but you also have to understand that as a Ruby programmer I strive to understand how Ruby works per se, that is my goal. And not being a committer, I cannot by myself know if the mismatches with the docs are intentional, an overlook, or if intentional, which is the mental model that fits that mismatch.

[fxn (Xavier Noria)]

You have also to take into account that in that example const_added is called, Module#constants lists :Foo, which is consistent with the autoload but not with autoload?.

So, the whole thing is kind of inconsistent right now.

But as a Ruby programmer, I cannot make sense of it without your help. Maybe the response of the Ruby team is "oh, that is right, `autoload? should be a fast lookup to the constants table and nothing else!", or "actually not, the way to make all this fit together is to revise such and such APIs".

[fxn (Xavier Noria)]

Finally, this ticket was intentionally not categorized as "Bug", but it is "Misc". Because I do not know if there is a bug.

[fxn (Xavier Noria)]

@mame (Yusuke Endoh) Let me put it all together:

Consider a script foo.rb:

autoload :Foo, 'foo' # could be a dynamic call
p autoload?(:Foo)    # could be a dynamic call

If we execute that with ruby -I. -rfoo -e1, we get nil. But this does not agree with the docs:

1. The constant Foo does not exist, therefore (according to the docs), the autoload is registered.
2. Module#constants lists :Foo, which is consistent.
3. Module#const_added is called with :Foo, which is consistent.

Now, the docs of Module#autoload? say:

Returns filename to be loaded if name is registered as autoload in the namespace of mod or one of its ancestors.

since the autoload is registered, I should get a filename back, not nil.

My question to the Ruby team is: Is the behavior OK and the docs should be updated, or should the implementation be updated instead?

My suggestion would be to keep the docs as they are and simplify autoload? to be a simple lookup in the constants table.

(Hope that explains it better.)

[mame (Yusuke Endoh)]

I understood as follows: You want @matz (Yukihiro Matsumoto) to clarify the behavior around autoload under circular require patterns, but you never want to explain your motivation example.

I said in #21035 that at the last dev meeting, the decision was made not to discuss anything by using code examples in the circular require pattern.
You are requesting that we discuss the same agenda item again with little new material. Frankly, such a request would only waste the time of the dev meeting and is not productive.

I don't understand why you are so concerned about circular require patterns. This is just my imagination, but you, the author of the zeitwerk, cannot avoid the presence of circular require patterns in the user code?
If so, for example, does it help you by issuing a warning or error for autoloading that would result in a circular require pattern?

[fxn (Xavier Noria)]

@mame (Yusuke Endoh) I am showing a registered autoload whose autoload? does not return a file name. If proves the documentation is wrong, or the implementation is wrong. But I do not know which is which. That is the point I am trying to make.

[fxn (Xavier Noria)]

It may also help to know that I am writing a book about Ruby constants, so it would be helpful to know if the documentation is wrong, or the implementation is wrong.

[fxn (Xavier Noria)]

@mame (Yusuke Endoh) I don't want anyone to waste their time, if you believe discussing this is so, please feel free to close.

I opened this ticket because the discussion went on in the previous one and @Eregon (Benoit Daloze), who is a Ruby committer, suggested to do so.

Why I don't show source code? Because this is not a ticket motivated by a need I have. Zeitwerk has been living with this edge case for 6 years. I am fine.

I thought that pointing out that the API is not consistent would be of your interest per se. Resolutions could be several:

• Oh, thank you, autoload? is OK, we could document it checks features.
• Oh, thank you, autoload should be the one checking features, actually, the piece that is not right here is autoload.
• Oh, thank you, autolad is fine, but autoload? should return the file name, let's fix that.

But we do not seem to be aligned. It is OK, please, feel free to close.

[mame (Yusuke Endoh)]

Why I don't show source code? Because this is not a ticket motivated by a need I have.

I see, you really do not have a use case. I finally understand.
I had misunderstood that you were hiding your use cases and somewhat challenging us.
(I am not sure if my English wording is appropriate here, sorry if it makes a wrong sense.)
Sorry for the frustration.

Let me explain why I am eager to hear the use cases.

In principle, all proposed changes to Ruby are considered based on the use cases that would require the change.
Even if it is just an inconsistency, we force ourselves to assume use cases that could be affected by the inconsistency and then consider how (and whether) to fix it.

Therefore, ticket discussions at the dev meetings begin with an understanding of the use case.
Without a background explanation in the ticket, a great deal of time is spent to try identifying use cases.
Still, it is impossible to cover all the use cases in a limited amount of time, so we could reach incomplete conclusions.
We sometimes end up finding a different use case than the one the proposer has in mind, leading to a conclusion that the users, including the proposer, do not want.
If a reasonable use case cannot be found, we need to ask the proposer to clarify, which takes time.
Or we may choose WONTFIX, or make a random choice based on matz's intuition, which is also often not the result desired by the proposer.

You may think that Ruby committers has the "answer" to the question you have and you want to just get it out of us.
In fact, we have no "answer". At least I do not have. I believe matz does not have it either for your question yet.
We need to decide how (and whether) to fix it by using the use cases.
If there is no particular use case that is troubling you, then we tend to choose "don't touch it" or just rely on matz's intuition.

As for autoload, there is a long history of subtle and repeated fine-tuning based on problems encountered in real-workd use cases.
Therefore, I feel strong resistance to changing something based on intuition alone.
I am afraid that by making a bad change, Zeitwerk stops working, which will affect many people.

This is the reason why I insisted on asking you, the author of Zeitwerk, for background on this issue.

[fxn (Xavier Noria)]

Oohhh, now I understand! Thanks for explaining this.

I am also not familiar with how dev meetings are run, so we were coming from different angles to this and I guess we were not quite understanding each other in the threads.

Let me think if it is worth explaining how usage of autoload? in Zeitwerk is affected by this (the gem has a workaround to deal with this behavior).

Let me think, I'll write back, thanks again for your clarifications @mame (Yusuke Endoh).

[fxn (Xavier Noria)]

Hi @mame (Yusuke Endoh).

My main motivation for starting this discussion was to 1) understand Ruby better, 2) bring some inconsistencies to the attention of the Ruby team and ask for clarification (here, I assumed the team knows autoload inside out and matz would instantly know the correct mental model/logic, your reply yesterday helped me understand this assumption is not entirely correct) 3) know what to write in my book.

Now that I understand better how dev meetings work and how issues are evaluated, I am not sure the use case in Zeitwerk affected by this behavior is worth a debate.

Lastly, let me address this sentence:

I had misunderstood that you were hiding your use cases and somewhat challenging us.

You can be 100% confident I'd never do that.

My questions & discussions are in good faith and with an OSS collaboration spirit.

I have a huge respect for everybody in the Ruby team. When I wrote Zeitwerk, I told Matz that if he wanted to keep the deprecation of autoload I'd cancel the project, and I meant it. Suggested to Matz in Sarajevo last year that if he wanted to remove autoload from Ruby and think of an alternative, I'd be OK and willing to help.

We may agree or disagree in specific technical details, that is normal, but always with good intention!

I think we can close, thanks @mame (Yusuke Endoh)!

[jeremyevans0 (Jeremy Evans)]

• Status changed from Open to Closed