Project

General

Profile

Actions

Bug #16680

closed

Symlink folder in $LOAD_PATH does not work with autoload

Added by zw963 (Wei Zheng) about 4 years ago. Updated almost 3 years ago.

Status:
Closed
Assignee:
-
Target version:
-
[ruby-core:97410]

Description

The following is a full reproducible process. I will use a gem named 'looksee' to reproduce this.

  1. Do gem install looksee

  2. Go to gem folder and copy lib content into /tmp/test_looksee so that we have a folder like this:

     ╰─ $ tree /tmp/test_looksee
    /tmp/test_looksee
    └── lib
        ├── looksee
        │   ├── adapter
        │   │   ├── base.rb
        │   │   └── rubinius.rb
        │   ├── adapter.rb
        │   ├── clean.rb
        │   ├── columnizer.rb
        │   ├── core_ext.rb
        │   ├── editor.rb
        │   ├── help.rb
        │   ├── inspector.rb
        │   ├── lookup_path.rb
        │   ├── mri.so
        │   └── version.rb
        └── looksee.rb
    
  3. Create a new symlink to this folder.

     ╰─ $ln -s /tmp/test_looksee /tmp/test_looksee1
    
     ╰─ $ ls -alhd /tmp/test_looksee*
    drwxr-xr-x 3 zw963 zw963 60 2020-03-08 01:39 /tmp/test_looksee/
    lrwxrwxrwx 1 zw963 zw963 13 2020-03-08 02:08 /tmp/test_looksee1 -> 
    test_looksee//
    
  4. Run the following command to reproduce this issue.

     ╰─ $ ruby -I/tmp/test_looksee1/lib/ -rlooksee -e 'puts 100'
    Traceback (most recent call last):
            10: from /home/zw963/others/.rvm/rubies/ruby-2.7.0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
         9: from /home/zw963/others/.rvm/rubies/ruby-2.7.0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
         8: from /tmp/test_looksee1/lib/looksee.rb:1:in `<top (required)>'
         7: from /home/zw963/others/.rvm/rubies/ruby-2.7.0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
         6: from /home/zw963/others/.rvm/rubies/ruby-2.7.0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
         5: from /tmp/test_looksee1/lib/looksee/clean.rb:4:in `<top (required)>'
         4: from /tmp/test_looksee1/lib/looksee/clean.rb:171:in `<module:Looksee>'
         3: from /home/zw963/others/.rvm/rubies/ruby-2.7.0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
         2: from /home/zw963/others/.rvm/rubies/ruby-2.7.0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
         1: from /tmp/test_looksee1/lib/looksee/adapter.rb:1:in `<top (required)>'
    

/tmp/test_looksee1/lib/looksee/adapter.rb:2:in `module:Looksee': uninitialized constant Looksee::Adapter (NameError)
```

You may want to ask why use symlinks folder in $LOAD_PATH. I think this is a quite common case; many ruby developers have a local gem not managed by Rubygems or Bundler. They just use $LOAD_PATH to require/load it, and it is hard to ensure everyone's folder is not a symlink.

I think this is a breaking change. All those codes worked quite well before 2.7 because old codes always
follow symlinks. The following is an example from ruby 2.6.3, which works.

 ╰─ $ ruby -I/tmp/test_looksee1/lib/ -rlooksee -e 'puts 100'
Traceback (most recent call last):
        8: from /home/zw963/others/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
        7: from /home/zw963/others/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
        6: from /tmp/test_looksee/lib/looksee.rb:1:in `<top (required)>'
        5: from /home/zw963/others/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
        4: from /home/zw963/others/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
        3: from /tmp/test_looksee/lib/looksee/clean.rb:4:in `<top (required)>'
        2: from /tmp/test_looksee/lib/looksee/clean.rb:171:in `<module:Looksee>'
        1: from /home/zw963/others/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
/home/zw963/others/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require': incompatible library version - /tmp/test_looksee/lib/looksee/mri.so (LoadError)

As we can seen, those breaking codes come from autoload :Adapter, 'looksee/adapter'. I suspect
maybe this is caused by nested autoload. But this is only a guess, please check. Thank you.


Files

clipboard-202003080224-xglcq.png (183 KB) clipboard-202003080224-xglcq.png zw963 (Wei Zheng), 03/07/2020 06:23 PM

Related issues 1 (0 open1 closed)

Related to Ruby master - Bug #16462: Ruby 2.7 autoload not working with $RUBYLIB (maybe circular dependency error)ClosedActions
Actions #1

Updated by hsbt (Hiroshi SHIBATA) about 4 years ago

  • Related to Bug #16462: Ruby 2.7 autoload not working with $RUBYLIB (maybe circular dependency error) added

Updated by zw963 (Wei Zheng) almost 4 years ago

Sorry, any update on this?

Updated by Eregon (Benoit Daloze) almost 4 years ago

Maybe a solution is to always resolve symlinks in the cache of $LOAD_PATH?
One issue is that $LOAD_PATH entries might not necessarily exist.

Updated by zw963 (Wei Zheng) almost 4 years ago

Eregon (Benoit Daloze) wrote in #note-3:

One issue is that $LOAD_PATH entries might not necessarily exist.

Sorry, not really understood what your means about reference comment.

I think maybe it acceptable for only support non-symlink $LOAD_PATH when update to ruby 2.7.
but, it should be mentions on release change log and mention why we change this behavior
before ruby 2.7 release.

Updated by nobu (Nobuyoshi Nakada) almost 4 years ago

I can't reproduce it with the master.

$ ruby -v
ruby 2.8.0dev (2020-06-13T14:33:40Z master 2496bdb28f) [x86_64-linux]

First, copied looksee/lib directory to /tmp/test_looksee, and linked it to /tmp/test_looksee1, correct?

$ ls -lRA /tmp/test_looksee*
lrwxrwxrwx 1 ubuntu ubuntu   17 Jun 14 07:49 /tmp/test_looksee1 -> /tmp/test_looksee/

/tmp/test_looksee:
total 4
drwxr-xr-x 3 ubuntu ubuntu 4096 Jun 14 07:55 lib/

/tmp/test_looksee/lib:
total 8
drwxr-xr-x 3 ubuntu ubuntu 4096 Jun 14 07:42 looksee/
-rw-r--r-- 1 ubuntu ubuntu   51 Jun 14 07:42 looksee.rb

/tmp/test_looksee/lib/looksee:
total 44
drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 14 07:42 adapter/
-rw-r--r-- 1 ubuntu ubuntu  246 Jun 14 07:42 adapter.rb
-rw-r--r-- 1 ubuntu ubuntu 5276 Jun 14 07:42 clean.rb
-rw-r--r-- 1 ubuntu ubuntu 2107 Jun 14 07:42 columnizer.rb
-rw-r--r-- 1 ubuntu ubuntu  732 Jun 14 07:42 core_ext.rb
-rw-r--r-- 1 ubuntu ubuntu 1522 Jun 14 07:42 editor.rb
-rw-r--r-- 1 ubuntu ubuntu 1875 Jun 14 07:42 help.rb
-rw-r--r-- 1 ubuntu ubuntu 2046 Jun 14 07:42 inspector.rb
-rw-r--r-- 1 ubuntu ubuntu 2292 Jun 14 07:42 lookup_path.rb
-rw-r--r-- 1 ubuntu ubuntu  128 Jun 14 07:42 version.rb

/tmp/test_looksee/lib/looksee/adapter:
total 8
-rw-r--r-- 1 ubuntu ubuntu 2018 Jun 14 07:42 base.rb
-rw-r--r-- 1 ubuntu ubuntu  629 Jun 14 07:42 rubinius.rb

Second, run with the real path, and got a LoadError on mri.so as expected.

$ ruby -I/tmp/test_looksee/lib/ -rlooksee -e 'puts 100'
/home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require': cannot load such file -- looksee/mri.so (LoadError)
	from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	from /tmp/test_looksee/lib/looksee/clean.rb:171:in `<module:Looksee>'
	from /tmp/test_looksee/lib/looksee/clean.rb:4:in `<top (required)>'
	from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	from /tmp/test_looksee/lib/looksee.rb:1:in `<top (required)>'
	from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
bash: exit 1

Then, tried the symlink path, got the same error.

$ ruby -I/tmp/test_looksee1/lib/ -rlooksee -e 'puts 100'
/home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require': cannot load such file -- looksee/mri.so (LoadError)
	from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	from /tmp/test_looksee/lib/looksee/clean.rb:171:in `<module:Looksee>'
	from /tmp/test_looksee/lib/looksee/clean.rb:4:in `<top (required)>'
	from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	from /tmp/test_looksee/lib/looksee.rb:1:in `<top (required)>'
	from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
	from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
bash: exit 1

Updated by nobu (Nobuyoshi Nakada) almost 4 years ago

I could reproduce it with 2.7.1 and 2.7 head.

Updated by zw963 (Wei Zheng) over 3 years ago

nobu (Nobuyoshi Nakada) wrote in #note-6:

I can't reproduce it with the master.

I could reproduce it with 2.7.1 and 2.7 head.

Cool, good news, so, it was fixed on our master, right?

Actions #8

Updated by sawa (Tsuyoshi Sawada) over 3 years ago

  • Subject changed from [Breaking Change] Ruby 2.7 not support symlinks folder in $LOAD_PATH to work with autoload. to Symlink folder in $LOAD_PATH does not work with autoload
  • Description updated (diff)

Updated by jeremyevans0 (Jeremy Evans) almost 3 years ago

  • Status changed from Open to Closed
  • Backport changed from 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN to 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: REQUIRED

I tried this with Ruby 2.6, 2.7, and 3.0, and got the same results with all three:

$ ls -ld /home/jeremy/tmp/looksee1
lrwxr-xr-x  1 jeremy  jeremy  7 Apr 28 13:19 /home/jeremy/tmp/looksee1 -> looksee
$ ruby26 -I /home/jeremy/tmp/looksee1/lib -r looksee -e 'puts 100'
/usr/local/lib/ruby/2.6/rubygems/core_ext/kernel_require.rb:54:in `require': cannot load such file -- looksee/mri.so (LoadError)
        from /usr/local/lib/ruby/2.6/rubygems/core_ext/kernel_require.rb:54:in `require'
        from /home/jeremy/tmp/looksee/lib/looksee/clean.rb:171:in `<module:Looksee>'
        from /home/jeremy/tmp/looksee/lib/looksee/clean.rb:4:in `<top (required)>'
        from /usr/local/lib/ruby/2.6/rubygems/core_ext/kernel_require.rb:54:in `require'
        from /usr/local/lib/ruby/2.6/rubygems/core_ext/kernel_require.rb:54:in `require'
        from /home/jeremy/tmp/looksee/lib/looksee.rb:1:in `<top (required)>'
        from /usr/local/lib/ruby/2.6/rubygems/core_ext/kernel_require.rb:54:in `require'
        from /usr/local/lib/ruby/2.6/rubygems/core_ext/kernel_require.rb:54:in `require'
$ ruby27 -I /home/jeremy/tmp/looksee1/lib -r looksee -e 'puts 100'
Traceback (most recent call last):
        8: from /usr/local/lib/ruby/2.7/rubygems/core_ext/kernel_require.rb:83:in `require'
        7: from /usr/local/lib/ruby/2.7/rubygems/core_ext/kernel_require.rb:83:in `require'
        6: from /home/jeremy/tmp/looksee/lib/looksee.rb:1:in `<top (required)>'
        5: from /usr/local/lib/ruby/2.7/rubygems/core_ext/kernel_require.rb:83:in `require'
        4: from /usr/local/lib/ruby/2.7/rubygems/core_ext/kernel_require.rb:83:in `require'
        3: from /home/jeremy/tmp/looksee/lib/looksee/clean.rb:4:in `<top (required)>'
        2: from /home/jeremy/tmp/looksee/lib/looksee/clean.rb:171:in `<module:Looksee>'
        1: from /usr/local/lib/ruby/2.7/rubygems/core_ext/kernel_require.rb:83:in `require'
/usr/local/lib/ruby/2.7/rubygems/core_ext/kernel_require.rb:83:in `require': cannot load such file -- looksee/mri.so (LoadError)
$ ruby30 -I /home/jeremy/tmp/looksee1/lib -r looksee -e 'puts 100'
<internal:/usr/local/lib/ruby/3.0/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- looksee/mri.so (LoadError)
        from <internal:/usr/local/lib/ruby/3.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from /home/jeremy/tmp/looksee/lib/looksee/clean.rb:171:in `<module:Looksee>'
        from /home/jeremy/tmp/looksee/lib/looksee/clean.rb:4:in `<top (required)>'
        from <internal:/usr/local/lib/ruby/3.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from <internal:/usr/local/lib/ruby/3.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from /home/jeremy/tmp/looksee/lib/looksee.rb:1:in `<top (required)>'
        from <internal:/usr/local/lib/ruby/3.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from <internal:/usr/local/lib/ruby/3.0/rubygems/core_ext/kernel_require.rb>:85:in `require'

I tried with both the master branch and the v4.2.0 tag for looksee with the same results, just in case looksee had worked around this problem another way.

So I think this problem has been resolved by Ruby 2.7.3. If you can still recreate the issue on Ruby 2.7.3, please bisect to find the failing commit and fixing commit, and note it in this ticket, and we can mark this for backporting.

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0Like0