Bug #15469
closedRuby2.6 included `bundler` does not handle specified `csv` gem.
Description
I want to use csv
library which is same version of Ruby 2.5.
Because Ruby 2.6's csv library has some breaking changes and they break our product.
However, Ruby 2.6's bundler
does not handle csv
which I specified the version.
Test code¶
- Gemfile
source "https://rubygems.org"
gem 'roo'
gem 'csv', '1.0.0'
- test.rb
require "bundler/setup"
require "csv"
p CSV::VERSION
Result¶
$ bundle exec ruby -v test.rb
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-darwin18]
"3.0.2"
CSV::VERSION
shows 3.0.2
and it indicates that Ruby 2.6 included csv
library was used.
Workaround¶
If I install the bundler
gem through gem install bundler
, it works well.
$ gem install bundler
Fetching bundler-1.17.3.gem
Successfully installed bundler-1.17.3
1 gem installed
$ bundle exec ruby -v test.rb
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-darwin18]
"2.4.8"
Files
Updated by hsbt (Hiroshi SHIBATA) almost 6 years ago
- Status changed from Open to Assigned
- Assignee set to hsbt (Hiroshi SHIBATA)
What's the version of csv library that is you expected?
I wonder csv-1.0.0 on your Gemfile.
Updated by watson1978 (Shizuo Fujita) almost 6 years ago
I need that CSV::VERSION indicates 2.4.8 as well as Workaround.
Updated by tad (Tadashi Saito) almost 6 years ago
I could reproduce too with "Test code".
Also, I could reproduce with simpler code below.
Gemfile¶
source "https://rubygems.org"
gem "csv", "1.0.0"
test.rb¶
require "csv"
p csv: CSV::VERSION
results¶
$ ruby2.6 -v && bundle2.6 -v
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-linux]
Bundler version 1.17.2
$ bundle2.6 exec ruby2.6 test.rb
{:csv=>"3.0.2"} # expects "2.4.8" for 1.0.0 gem https://github.com/ruby/csv/blob/v1.0.0/lib/csv.rb#L211
In addition, it does not seem to be limited to "csv" gem.
I confirmed that the same problem occurs with "json".
Updated by aeroastro (Takumasa Ochi) almost 6 years ago
I have found out the root cause of this.
Root Cause¶
- Bundler has become a built-in gem since Ruby 2.6.
-
$LOAD_PATH
required for bundler is moved from custom gem path ( e.g.lib/ruby/gems/2.5.0/gems/bundler-1.17.2/lib
) to ruby built-in gem path (lib/ruby/2.6.0
) - In the same path,
csv
,json
, and other built-in gems are located. - When invoked via
bundler
, this path forbundler
is inserted at the head of$LOAD_PATH
, which takes precedence over all the other paths. - These built-in gems are loaded instead of those specified by
Gemfile
.
Why Workaround Works¶
When bundler is updated by gem install
, the $LOAD_PATH
is moved from ruby built-in gem path to custom gem path.
This fixes shared built-in gem path issue.
Suggested Fix¶
IMHO, moving bundler to isolated path instead of shared built-in path would fix this issue.
Bug Detail¶
Following are the code to reproduce and explain this issue.
Gemfile¶
source "https://rubygems.org"
gem 'json', '~> 1.8.6'
Code¶
before = $LOADED_FEATURES.dup
require 'json'
after = $LOADED_FEATURES.dup
require 'yaml'
puts YAML.dump(
ruby: RUBY_VERSION,
bundler: (Bundler::VERSION rescue nil),
json: JSON::VERSION,
load_path: $LOAD_PATH,
added_features: after - before,
)
JSON.dump("GET")
Result¶
# I have installed ruby with rbenv and ruby-build
# `bundle install --path=vendor/bundle` is executed as prerequisites
$ rbenv shell 2.5.3 && bundle exec ruby test.rb
---
:ruby: 2.5.3
:bundler: 1.17.2
:json: 1.8.6
:load_path:
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/gems/bundler-1.17.2/lib"
- "/Users/user-name/.rbenv/rbenv.d/exec/gem-rehash"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/extensions/x86_64-darwin-16/2.5.0-static/json-1.8.6"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/gems/bundler-1.17.2/lib/gems/bundler-1.17.2/lib"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/site_ruby/2.5.0"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/site_ruby/2.5.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/site_ruby"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/vendor_ruby/2.5.0"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/vendor_ruby/2.5.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/vendor_ruby"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/2.5.0"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/2.5.0/x86_64-darwin16"
:added_features:
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib/json/version.rb"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/2.5.0/ostruct.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib/json/generic_object.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib/json/common.rb"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/2.5.0/x86_64-darwin16/enc/utf_16be.bundle"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/2.5.0/x86_64-darwin16/enc/utf_16le.bundle"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/2.5.0/x86_64-darwin16/enc/utf_32be.bundle"
- "/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/2.5.0/x86_64-darwin16/enc/utf_32le.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib/json/ext/parser.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib/json/ext/generator.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib/json/ext.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib/json.rb"
$ rbenv shell 2.6.0 && bundle exec ruby test.rb
---
:ruby: 2.6.0
:bundler: 1.17.2
:json: 2.1.0
:load_path:
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0"
- "/Users/user-name/.rbenv/rbenv.d/exec/gem-rehash"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/extensions/x86_64-darwin-16/2.6.0-static/json-1.8.6"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/gems/2.6.0/gems/bundler-1.17.2/lib"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/site_ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/site_ruby/2.6.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/site_ruby"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/vendor_ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/vendor_ruby/2.6.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/vendor_ruby"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16"
:added_features:
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json/version.rb"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/ostruct.rb"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json/generic_object.rb"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json/common.rb"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_16be.bundle"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_16le.bundle"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_32be.bundle"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_32le.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext/parser.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext/generator.bundle"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json/ext.rb"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json.rb"
Traceback (most recent call last):
3: from test.rb:15:in `<main>'
2: from /Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json/common.rb:394:in `dump'
1: from /Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json/common.rb:224:in `generate'
/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/json/common.rb:224:in `generate': only generation of JSON objects or arrays allowed (JSON::GeneratorError)
$ gem install bundler -v 1.17.3
Fetching bundler-1.17.3.gem
Successfully installed bundler-1.17.3
Parsing documentation for bundler-1.17.3
Installing ri documentation for bundler-1.17.3
Done installing documentation for bundler after 3 seconds
1 gem installed
$ rbenv shell 2.6.0 && bundle exec ruby test.rb
---
:ruby: 2.6.0
:bundler: 1.17.3
:json: 1.8.6
:load_path:
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib"
- "/Users/user-name/.rbenv/rbenv.d/exec/gem-rehash"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/extensions/x86_64-darwin-16/2.6.0-static/json-1.8.6"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/gems/bundler-1.17.3/lib"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/site_ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/site_ruby/2.6.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/site_ruby"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/vendor_ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/vendor_ruby/2.6.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/vendor_ruby"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16"
:added_features:
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/version.rb"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/ostruct.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/generic_object.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/common.rb"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_16be.bundle"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_16le.bundle"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_32be.bundle"
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_32le.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext/parser.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext/generator.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json.rb"
You can see the first element in $LOAD_PATH
is problematic when built-in bundler
is used. ( /Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0
)
json
in shared built-in gem path is required instead of Gemfile-specified json
gem.
(The exception above is caused by mixed versions of json
gem.)
Updated by Eregon (Benoit Daloze) almost 6 years ago
Should Bundler be a bundled gem (and not a default gem)?
I think this would help a lot, and allow uninstalling the preinstalled bundler if needed.
As mentioned above, it would also keep the Bundler implementation simple by being able to assume it's in a gem-like layout.
Updated by hsbt (Hiroshi SHIBATA) almost 6 years ago
- Related to Bug #15500: Behavior of require method in 2.5 is different from 2.4 and 2.6 added
Updated by hsbt (Hiroshi SHIBATA) almost 6 years ago
Can you try this with Ruby 2.6.1? r66867 fixed this issue maybe.
Updated by watson1978 (Shizuo Fujita) almost 6 years ago
hsbt (Hiroshi SHIBATA) wrote:
Can you try this with Ruby 2.6.1? r66867 fixed this issue maybe.
Thank you for your working.
I re-tried the test code in description,
However, there is not difference for me....
$ bundle install
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
Using bundler 1.17.2
Fetching rubyzip 1.2.2
Fetching csv 1.0.0
Fetching mini_portile2 2.4.0
Installing csv 1.0.0
Installing mini_portile2 2.4.0
Installing rubyzip 1.2.2
Fetching nokogiri 1.10.1
Installing nokogiri 1.10.1 with native extensions
Fetching roo 2.8.1
Installing roo 2.8.1
Bundle complete! 2 Gemfile dependencies, 6 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
$ bundle exec ruby -v test.rb
ruby 2.6.1p33 (2019-01-30 revision 66950) [x86_64-darwin18]
"3.0.4"
Updated by hsbt (Hiroshi SHIBATA) almost 6 years ago
Thanks all.
I investigate this issue with https://bugs.ruby-lang.org/issues/15469#note-4. and fixed this issue on Bundler::SharedHelper.set_rubylib
.
Does anyone try with the attached patch?
Updated by watson1978 (Shizuo Fujita) almost 6 years ago
Does anyone try with the attached patch?
I've got an expected result finally with the patch.
Thank you for great works!!!
$ bundle exec ruby -v test.rb
ruby 2.6.1p33 (2019-01-30 revision 66950) [x86_64-darwin18]
"2.4.8"
$ ruby -v test.rb
ruby 2.6.1p33 (2019-01-30 revision 66950) [x86_64-darwin18]
"2.4.8"
Updated by hsbt (Hiroshi SHIBATA) almost 6 years ago
- Backport changed from 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN to 2.4: DONTNEED, 2.5: DONTNEED, 2.6: REQUIRED
Updated by aeroastro (Takumasa Ochi) over 5 years ago
Does anyone try with the attached patch?
I've also got the ideal result as follows. (with patched ruby 2.6.1)
Now there is no issue on $LOAD_PATH
.
Thank you very much for your great work!
$ rbenv shell 2.6.1 && ruby --version && bundle exec ruby test.rb
ruby 2.6.1p33 (2019-01-30 revision 66950) [x86_64-darwin16]
---
:ruby: 2.6.1
:bundler: 1.17.2
:json: 1.8.6
:load_path:
- "/Users/user-name/.rbenv/rbenv.d/exec/gem-rehash"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/extensions/x86_64-darwin-16/2.6.0-static/json-1.8.6"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/bundler-1.17.2/lib"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/site_ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/site_ruby/2.6.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/site_ruby"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/vendor_ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/vendor_ruby/2.6.0/x86_64-darwin16"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/vendor_ruby"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/2.6.0"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/2.6.0/x86_64-darwin16"
:added_features:
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/version.rb"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/2.6.0/ostruct.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/generic_object.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/common.rb"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_16be.bundle"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_16le.bundle"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_32be.bundle"
- "/Users/user-name/.rbenv/versions/2.6.1/lib/ruby/2.6.0/x86_64-darwin16/enc/utf_32le.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext/parser.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext/generator.bundle"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json/ext.rb"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib/json.rb"
Updated by hsbt (Hiroshi SHIBATA) over 5 years ago
- Status changed from Assigned to Closed
Applied in changeset trunk|r67006.
Ignore to add bundler lib direcotry if it is same as rubylibdir.
[Bug #15469][ruby-core:90742]
Updated by hsbt (Hiroshi SHIBATA) over 5 years ago
I attached the patch of the upstream fixes.
The bundler team will ship Bundler 1.7.4/2.0.2 with this patch.
Updated by naruse (Yui NARUSE) over 5 years ago
- Backport changed from 2.4: DONTNEED, 2.5: DONTNEED, 2.6: REQUIRED to 2.4: DONTNEED, 2.5: DONTNEED, 2.6: DONE
ruby_2_6 r67179 merged revision(s) 67006.