Project

General

Profile

Actions

Bug #15469

closed

Ruby2.6 included `bundler` does not handle specified `csv` gem.

Added by watson1978 (Shizuo Fujita) over 5 years ago. Updated about 5 years ago.

Status:
Closed
Target version:
-
[ruby-core:90742]

Description

I want to use csv library which is same version of Ruby 2.5.
Because Ruby 2.6's csv library has some breaking changes and they break our product.

However, Ruby 2.6's bundler does not handle csv which I specified the version.

Test code

  • Gemfile
source "https://rubygems.org"

gem 'roo'
gem 'csv', '1.0.0'
  • test.rb
require "bundler/setup"
require "csv"

p CSV::VERSION

Result

$ bundle exec ruby -v test.rb
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-darwin18]
"3.0.2"

CSV::VERSION shows 3.0.2 and it indicates that Ruby 2.6 included csv library was used.

Workaround

If I install the bundler gem through gem install bundler, it works well.

$ gem install bundler
Fetching bundler-1.17.3.gem
Successfully installed bundler-1.17.3
1 gem installed

$ bundle exec ruby -v test.rb
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-darwin18]
"2.4.8"

Files

15469-bundler-loadpath.patch (528 Bytes) 15469-bundler-loadpath.patch hsbt (Hiroshi SHIBATA), 02/02/2019 08:29 AM
r15469-bundler-final.patch (1.69 KB) r15469-bundler-final.patch hsbt (Hiroshi SHIBATA), 02/05/2019 06:14 AM

Related issues 1 (0 open1 closed)

Related to Ruby master - Bug #15500: Behavior of require method in 2.5 is different from 2.4 and 2.6Closedhsbt (Hiroshi SHIBATA)Actions

Updated by hsbt (Hiroshi SHIBATA) about 5 years ago

  • Status changed from Open to Assigned
  • Assignee set to hsbt (Hiroshi SHIBATA)

What's the version of csv library that is you expected?

I wonder csv-1.0.0 on your Gemfile.

Updated by watson1978 (Shizuo Fujita) about 5 years ago

I need that CSV::VERSION indicates 2.4.8 as well as Workaround.

Updated by tad (Tadashi Saito) about 5 years ago

I could reproduce too with "Test code".

Also, I could reproduce with simpler code below.

Gemfile

source "https://rubygems.org"

gem "csv", "1.0.0"

test.rb

require "csv"

p csv: CSV::VERSION

results

$ ruby2.6 -v && bundle2.6 -v
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-linux]
Bundler version 1.17.2
$ bundle2.6 exec ruby2.6 test.rb 
{:csv=>"3.0.2"} # expects "2.4.8" for 1.0.0 gem https://github.com/ruby/csv/blob/v1.0.0/lib/csv.rb#L211

In addition, it does not seem to be limited to "csv" gem.
I confirmed that the same problem occurs with "json".

Updated by aeroastro (Takumasa Ochi) about 5 years ago

I have found out the root cause of this.

Root Cause

  • Bundler has become a built-in gem since Ruby 2.6.
  • $LOAD_PATH required for bundler is moved from custom gem path ( e.g. lib/ruby/gems/2.5.0/gems/bundler-1.17.2/lib ) to ruby built-in gem path ( lib/ruby/2.6.0 )
  • In the same path, csv, json, and other built-in gems are located.
  • When invoked via bundler, this path for bundler is inserted at the head of $LOAD_PATH, which takes precedence over all the other paths.
  • These built-in gems are loaded instead of those specified by Gemfile.

Why Workaround Works

When bundler is updated by gem install, the $LOAD_PATH is moved from ruby built-in gem path to custom gem path.
This fixes shared built-in gem path issue.

Suggested Fix

IMHO, moving bundler to isolated path instead of shared built-in path would fix this issue.

Bug Detail

Following are the code to reproduce and explain this issue.

Gemfile

source "https://rubygems.org"

gem 'json', '~> 1.8.6'

Code

before = $LOADED_FEATURES.dup
require 'json'
after = $LOADED_FEATURES.dup

require 'yaml'

puts YAML.dump(
  ruby: RUBY_VERSION,
  bundler: (Bundler::VERSION rescue nil),
  json: JSON::VERSION,
  load_path: $LOAD_PATH,
  added_features: after - before,
)

JSON.dump("GET")

Result

# I have installed ruby with rbenv and ruby-build
# `bundle install --path=vendor/bundle` is executed as prerequisites

$ rbenv shell 2.5.3 && bundle exec ruby test.rb
---
:ruby: 2.5.3
:bundler: 1.17.2
:json: 1.8.6
:load_path:
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/gems/bundler-1.17.2/lib"
- "/Users/user-name/.rbenv/rbenv.d/exec/gem-rehash"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/extensions/x86_64-darwin-16/2.5.0-static/json-1.8.6"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/gems/bundler-1.17.2/lib/gems/bundler-1.17.2/lib"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/site_ruby/2.5.0"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/site_ruby/2.5.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/site_ruby"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/vendor_ruby/2.5.0"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/vendor_ruby/2.5.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/vendor_ruby"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/2.5.0"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/2.5.0/x86_64-darwin16"
:added_features:
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib/json/version.rb"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/2.5.0/ostruct.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib/json/generic_object.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib/json/common.rb"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/2.5.0/x86_64-darwin16/enc/utf_16be.bundle"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/2.5.0/x86_64-darwin16/enc/utf_16le.bundle"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/2.5.0/x86_64-darwin16/enc/utf_32be.bundle"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/2.5.0/x86_64-darwin16/enc/utf_32le.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib/json/ext/parser.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib/json/ext/generator.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib/json/ext.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib/json.rb"

$ rbenv shell 2.6.0 && bundle exec ruby test.rb
---
:ruby: 2.6.0
:bundler: 1.17.2
:json: 2.1.0
:load_path:
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0"
- "/Users/user-name/.rbenv/rbenv.d/exec/gem-rehash"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/extensions/x86_64-darwin-16/2.6.0-static/json-1.8.6"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/gems/2.6.0/gems/bundler-1.17.2/lib"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/site_ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/site_ruby/2.6.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/site_ruby"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/vendor_ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/vendor_ruby/2.6.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/vendor_ruby"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16"
:added_features:
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json/version.rb"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/ostruct.rb"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json/generic_object.rb"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json/common.rb"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_16be.bundle"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_16le.bundle"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_32be.bundle"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_32le.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext/parser.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext/generator.bundle"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json/ext.rb"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json.rb"
Traceback (most recent call last):
	3: from test.rb:15:in `<main>'
	2: from /Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json/common.rb:394:in `dump'
	1: from /Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json/common.rb:224:in `generate'
/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json/common.rb:224:in `generate': only generation of JSON objects or arrays allowed (JSON::GeneratorError)

$ gem install bundler -v 1.17.3
Fetching bundler-1.17.3.gem
Successfully installed bundler-1.17.3
Parsing documentation for bundler-1.17.3
Installing ri documentation for bundler-1.17.3
Done installing documentation for bundler after 3 seconds
1 gem installed

$ rbenv shell 2.6.0 && bundle exec ruby test.rb
---
:ruby: 2.6.0
:bundler: 1.17.3
:json: 1.8.6
:load_path:
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib"
- "/Users/user-name/.rbenv/rbenv.d/exec/gem-rehash"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/extensions/x86_64-darwin-16/2.6.0-static/json-1.8.6"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/gems/bundler-1.17.3/lib"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/site_ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/site_ruby/2.6.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/site_ruby"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/vendor_ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/vendor_ruby/2.6.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/vendor_ruby"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16"
:added_features:
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/version.rb"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/ostruct.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/generic_object.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/common.rb"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_16be.bundle"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_16le.bundle"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_32be.bundle"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_32le.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext/parser.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext/generator.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json.rb"

You can see the first element in $LOAD_PATH is problematic when built-in bundler is used. ( /Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0 )
json in shared built-in gem path is required instead of Gemfile-specified json gem.

(The exception above is caused by mixed versions of json gem.)

Updated by Eregon (Benoit Daloze) about 5 years ago

Should Bundler be a bundled gem (and not a default gem)?
I think this would help a lot, and allow uninstalling the preinstalled bundler if needed.

As mentioned above, it would also keep the Bundler implementation simple by being able to assume it's in a gem-like layout.

Actions #6

Updated by hsbt (Hiroshi SHIBATA) about 5 years ago

  • Related to Bug #15500: Behavior of require method in 2.5 is different from 2.4 and 2.6 added

Updated by hsbt (Hiroshi SHIBATA) about 5 years ago

@watson1978 (Shizuo Fujita)

Can you try this with Ruby 2.6.1? r66867 fixed this issue maybe.

Updated by watson1978 (Shizuo Fujita) about 5 years ago

hsbt (Hiroshi SHIBATA) wrote:

@watson1978 (Shizuo Fujita)

Can you try this with Ruby 2.6.1? r66867 fixed this issue maybe.

Thank you for your working.
I re-tried the test code in description,
However, there is not difference for me....

$ bundle install
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
Using bundler 1.17.2
Fetching rubyzip 1.2.2
Fetching csv 1.0.0
Fetching mini_portile2 2.4.0
Installing csv 1.0.0
Installing mini_portile2 2.4.0
Installing rubyzip 1.2.2
Fetching nokogiri 1.10.1
Installing nokogiri 1.10.1 with native extensions
Fetching roo 2.8.1
Installing roo 2.8.1
Bundle complete! 2 Gemfile dependencies, 6 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.

$ bundle exec ruby -v test.rb
ruby 2.6.1p33 (2019-01-30 revision 66950) [x86_64-darwin18]
"3.0.4"

Updated by hsbt (Hiroshi SHIBATA) about 5 years ago

Thanks all.

I investigate this issue with https://bugs.ruby-lang.org/issues/15469#note-4. and fixed this issue on Bundler::SharedHelper.set_rubylib.

Does anyone try with the attached patch?

Updated by watson1978 (Shizuo Fujita) about 5 years ago

Does anyone try with the attached patch?

I've got an expected result finally with the patch.
Thank you for great works!!!

$ bundle exec ruby -v test.rb
ruby 2.6.1p33 (2019-01-30 revision 66950) [x86_64-darwin18]
"2.4.8"

$ ruby -v test.rb
ruby 2.6.1p33 (2019-01-30 revision 66950) [x86_64-darwin18]
"2.4.8"
Actions #11

Updated by hsbt (Hiroshi SHIBATA) about 5 years ago

  • Backport changed from 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN to 2.4: DONTNEED, 2.5: DONTNEED, 2.6: REQUIRED

Updated by aeroastro (Takumasa Ochi) about 5 years ago

Does anyone try with the attached patch?

I've also got the ideal result as follows. (with patched ruby 2.6.1)
Now there is no issue on $LOAD_PATH.

Thank you very much for your great work!

$ rbenv shell 2.6.1 && ruby --version && bundle exec ruby test.rb
ruby 2.6.1p33 (2019-01-30 revision 66950) [x86_64-darwin16]
---
:ruby: 2.6.1
:bundler: 1.17.2
:json: 1.8.6
:load_path:
- "/Users/user-name/.rbenv/rbenv.d/exec/gem-rehash"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/extensions/x86_64-darwin-16/2.6.0-static/json-1.8.6"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/bundler-1.17.2/lib"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/site_ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/site_ruby/2.6.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/site_ruby"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/vendor_ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/vendor_ruby/2.6.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/vendor_ruby"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/2.6.0/x86_64-darwin16"
:added_features:
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/version.rb"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/2.6.0/ostruct.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/generic_object.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/common.rb"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_16be.bundle"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_16le.bundle"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_32be.bundle"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_32le.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext/parser.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext/generator.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json.rb"
Actions #13

Updated by hsbt (Hiroshi SHIBATA) about 5 years ago

  • Status changed from Assigned to Closed

Applied in changeset trunk|r67006.


Ignore to add bundler lib direcotry if it is same as rubylibdir.

[Bug #15469][ruby-core:90742]

Updated by hsbt (Hiroshi SHIBATA) about 5 years ago

I attached the patch of the upstream fixes.

The bundler team will ship Bundler 1.7.4/2.0.2 with this patch.

Updated by naruse (Yui NARUSE) about 5 years ago

  • Backport changed from 2.4: DONTNEED, 2.5: DONTNEED, 2.6: REQUIRED to 2.4: DONTNEED, 2.5: DONTNEED, 2.6: DONE

ruby_2_6 r67179 merged revision(s) 67006.

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0