Bug #9106
closed'gem install' doesn't copy .so files of ext libs
Description
'gem install' should copy files of 'ext/**/*.so' of each installed gems, but doesn't.
$ ruby -v
ruby 2.1.0dev (2013-11-12 trunk 43653) [x86_64-linux]
$ gem install msgpack
Fetching: msgpack-0.5.7.gem (100%)
Building native extensions. This could take a while...
Successfully installed msgpack-0.5.7
[]
Parsing documentation for msgpack-0.5.7
Installing ri documentation for msgpack-0.5.7
Done installing documentation for msgpack after 0 seconds
1 gem installed
$ ruby -rmsgpack -e ''
/home/user/local/ruby-2.1.0-trunk/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in require': cannot load such file -- /home/user/local/ruby-2.1.0-trunk/lib/ruby/gems/2.1.0/gems/msgpack-0.5.7/lib/msgpack/msgpack (LoadError) from /home/user/local/ruby-2.1.0-trunk/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in
require'
from /home/user/local/ruby-2.1.0-trunk/lib/ruby/gems/2.1.0/gems/msgpack-0.5.7/lib/msgpack.rb:8:in rescue in <top (required)>' from /home/user/local/ruby-2.1.0-trunk/lib/ruby/gems/2.1.0/gems/msgpack-0.5.7/lib/msgpack.rb:3:in
<top (required)>'
from /home/user/local/ruby-2.1.0-trunk/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135:in require' from /home/user/local/ruby-2.1.0-trunk/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135:in
rescue in require'
from /home/user/local/ruby-2.1.0-trunk/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:144:in `require'
$ ls -l local/ruby-2.1.0-trunk/lib/ruby/gems/2.1.0/gems/msgpack-0.5.7/lib/msgpack
total 4
-rw-r--r-- 1 edge-dev edge 42 Nov 13 15:43 version.rb
$ ls -l local/ruby-2.1.0-trunk/lib/ruby/gems/2.1.0/gems/msgpack-0.5.7/ext/msgpack/msgpack.so
-rwxr-xr-x 1 edge-dev edge 1390955 Nov 13 15:43 local/ruby-2.1.0-trunk/lib/ruby/gems/2.1.0/gems/msgpack-0.5.7/ext/msgpack/msgpack.so
Updated by sorah (Sorah Fukumori) almost 11 years ago
- Status changed from Open to Assigned
- Assignee set to drbrain (Eric Hodel)
Eric, could you handle this?
Updated by drbrain (Eric Hodel) almost 11 years ago
- Priority changed from 7 to Normal
Yes, I will investigate.
Updated by drbrain (Eric Hodel) almost 11 years ago
=begin
Ugh. This is a misfeature of msgpack, it should trust that RubyGems will do the right thing:
here = File.expand_path(File.dirname(FILE))
require File.join(here, 'msgpack', 'version')
begin
m = /(\d+.\d+)/.match(RUBY_VERSION)
ver = m[1]
require File.join(here, 'msgpack', ver, 'msgpack')
rescue LoadError
require File.join(here, 'msgpack', 'msgpack')
end
=end
Updated by hsbt (Hiroshi SHIBATA) almost 11 years ago
- Status changed from Assigned to Feedback
tagomoris: Thank you for your report. Can you investigate to msgpack gem?
Updated by drbrain (Eric Hodel) almost 11 years ago
- Status changed from Feedback to Assigned
- Backport changed from 1.9.3: UNKNOWN, 2.0.0: UNKNOWN to 1.9.3: DONTNEED, 2.0.0: DONTNEED
For backwards compatibility I will need to alter RubyGems before Ruby 2.1 (I haven't had time yet).
Future versions of RubyGems will not support this lack of trust, though.
Updated by tagomoris (Satoshi Tagomori) almost 11 years ago
hsbt (Hiroshi SHIBATA) wrote:
tagomoris: Thank you for your report. Can you investigate to msgpack gem?
I don't understand why msgpack requires with such code. So I'll tell it to @frsyuki (Sadayuki Furuhashi), original developer of msgpack.
Updated by frsyuki (Sadayuki Furuhashi) almost 11 years ago
Hi I'm @frsyuki (Sadayuki Furuhashi).
-
I don't understand which behavior of RubyGems should the code trust. It assumes that msgpack.so is placed into lib/msgpack/msgpack.so because ext/msgpack/extconf.rb includes this line:
create_makefile('msgpack/msgpack') -
Which fix is appropriate?:
a) require File.join(here, '..', 'ext', 'msgpack', 'msgpack')
b) add "ext" directory to require_paths in gemspec
c) require "msgpack/msgpack"
d) require "msgpack#{RbConfig::CONFIG["DLEXT"]}"
Updated by drbrain (Eric Hodel) almost 11 years ago
=begin
@frsyuki (Sadayuki Furuhashi): you should trust rubygems to make msgpack/msgpack.so accessible somewhere in the $LOAD_PATH so:
require 'msgpack/msgpack'
so option c is the best fix.
With RubyGems 2.2 the directory you gem install
to can be shared across multiple ruby versions because the extensions end up in $GEM_HOME/extensions///. RubyGems will add that directory to $LOAD_PATH so require can find files in it.
For msgpack, on OS X with ruby 2.1.0 this would be:
$GEM_HOME/extensions/x86_64-darwin-12/2.0.0/msgpack-0.5.7/msgpack/msgpack.bundle
I will figure out a backward compatible solution so older msgpack will still work for RubyGems 2.x. There is no reason for you to rush out a new release of msgpack.
=end
Updated by frsyuki (Sadayuki Furuhashi) almost 11 years ago
It sounds good improvement of RubyGems to manage binaries for each ruby API versions.
Your comment totally makes sense. I'll fix msgpack next version. Thank you!
Updated by luislavena (Luis Lavena) almost 11 years ago
Hello,
Here is a 2009 discussion about gems and multiple binaries (targeting different versions of Ruby):
http://rubyforge.org/pipermail/rubygems-developers/2009-April/004522.html
My alternative to that issue is what you can see in projects like sqlite3-ruby:
https://github.com/sparklemotion/sqlite3-ruby/blob/master/lib/sqlite3.rb#L1-L7
And others that follow the guidelines of rake-compiler to deal with these multiple binaries in a single gem.
Of course, this is to deal with gems that package binaries, gems that are compiled on installation, we let RubyGems do its job.
Updated by drbrain (Eric Hodel) almost 11 years ago
- Status changed from Assigned to Closed
- % Done changed from 0 to 100
This issue was solved with changeset r43714.
Satoshi, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.
-
lib/rubygems: Update to RubyGems master 6a3d9f9. Changes include:
Compatibly renamed Gem::DependencyResolver to Gem::Resolver.
Added support for git gems in gem.deps.rb and Gemfile.
Fixed resolver bugs.
-
test/rubygems: ditto.
-
lib/rubygems/LICENSE.txt: Updated to license from RubyGems trunk.
[ruby-trunk - Bug #9086] -
lib/rubygems/commands/which_command.rb: RubyGems now indicates
failure when any file is missing. [ruby-trunk - Bug #9004] -
lib/rubygems/ext/builder: Extensions are now installed into the
extension install directory and the first directory in the require
path from the gem. This allows backwards compatibility with msgpack
and other gems that calculate full require paths.
[ruby-trunk - Bug #9106]
Updated by tagomoris (Satoshi Tagomori) almost 11 years ago
This problem is reproduced on ruby 2.1.0-preview2 on Linux/OSX.
'msgpack' gem cannot be loaded (LoadError) and 'ext/msgpack/msgpack.so' (or .bundle) is not copied to 'lib/msgpack/msgpack.so'.
About other library (ex: cool.io), binary files are copied to 'lib' directory.
P.S. I cannot reopen this ticket.
Updated by nagachika (Tomoyuki Chikanaga) almost 11 years ago
- Status changed from Closed to Assigned
Updated by drbrain (Eric Hodel) almost 11 years ago
Hrm, I will check it.
Updated by drbrain (Eric Hodel) almost 11 years ago
I found my bug!
I didn't recursively copy the files.
Fix soon!
Updated by drbrain (Eric Hodel) almost 11 years ago
- Status changed from Assigned to Closed
This issue was solved with changeset r43953.
Satoshi, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.
-
lib/rubygems: Update to RubyGems master baa965b. Notable changes:
Copy directories to lib/ when installing extensions. This completes
the fix for [ruby-trunk - Bug #9106] -
test/rubygems: ditto.