Bug #3140: gem activation has changed between 1.8 and 1.9

• Status: Closed
• Assignee: nobu (Nobuyoshi Nakada)
• Target version: 1.9.2
• ruby -v: ruby 1.9.2dev (2010-04-12 trunk 27317) [x86_64-darwin10.2.0]

[tenderlovemaking (Aaron Patterson)]

=begin
1.8 will raise gem activation errors where ruby 1.9 will not.

To reproduce this bug, first install these gems:

$ gem install rubygems-bug-parent
$ gem install rubygems-bug-child

Your gem list should look like this:

$ gem list rubygems-bug

*** LOCAL GEMS ***

rubygems-bug-child (1.1, 1.0)
rubygems-bug-parent (1.0)

Then run the following program:

 $ ruby -rubygems -e "require 'rubygems-bug-child'; require 'rubygems-bug-parent'"

Ruby 1.8 will raise an activation error because of the conflicting versions:

Here I am in version 1.1
/Library/Ruby/Site/1.8/rubygems.rb:230:in activate': can't activate rubygems-bug-child (= 1.0.0, runtime) for ["rubygems-bug-parent-1.0"], already activated rubygems-bug-child-1.1 for [] (Gem::LoadError) from /Library/Ruby/Site/1.8/rubygems.rb:246:in activate'
from /Library/Ruby/Site/1.8/rubygems.rb:245:in each' from /Library/Ruby/Site/1.8/rubygems.rb:245:in activate'
from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:35:in `require'
from -e:1

Ruby 1.9 will continue along using the 1.1 version, and never detect the version conflict:

$ ruby -rubygems -e "require 'rubygems-bug-child'; require 'rubygems-bug-parent'"
Here I am in version 1.1
=end

[zenspider (Ryan Davis)]

=begin

On Apr 14, 2010, at 20:36 , Rich Kilmer wrote:

1. Make RubyGems packaged files accessible with require/load in 1.9
   without having to require 'rubygems'
2. Minimize startup time (don't load the full rubygems library if its
   not going to be used)
3. Load rubygems proper if need be.

Is 1 a real requirement tho? I'm not sure it is. If it isn't, moving gem_prelude to something like 'rubygems/fast' would allow users a clean and easy choice of: a) nothing at all, leave me alone; b) rubygems proper; c) rubygems via the gem_prelude strategy.

You can even no-op rubygems/fast if defined? Gem, so it can be overridden via the commandline in case of a special conflict.

=end

[nobu (Nobuyoshi Nakada)]

=begin
Hi,

At Fri, 16 Apr 2010 03:08:27 +0900,
Evan Phoenix wrote in [ruby-core:29537]:

This moves the RubyGem custom require into the prelude and
triggers the full loading of RubyGems itself if the normal
require raises a LoadError. It fixes the problem because the
highest version gems are no longer pushed on $LOAD_PATH,
allowing RubyGems to activate gems and the proper
dependencies.

I don't like the duplication, so changed to keep #require
staying in custom_require.rb and introduce Gem.try_activate.

diff --git c/common.mk i/common.mk
index 36174c3..30b31c8 100644
--- c/common.mk
+++ i/common.mk
@@ -733,7 +733,9 @@ known_errors.inc: $(srcdir)/template/known_errors.inc.tmpl $(srcdir)/defs/known_
miniprelude.c: $(srcdir)/tool/compile_prelude.rb $(srcdir)/prelude.rb
$(BASERUBY) -I$(srcdir) $(srcdir)/tool/compile_prelude.rb $(srcdir)/prelude.rb $@

-prelude.c: $(srcdir)/tool/compile_prelude.rb $(RBCONFIG) $(srcdir)/lib/rubygems/defaults.rb $(PRELUDE_SCRIPTS) $(PREP)
+prelude.c: $(srcdir)/tool/compile_prelude.rb $(RBCONFIG) \

• $(srcdir)/lib/rubygems/defaults.rb $(srcdir)/lib/rubygems/custom_require.rb \
  

• $(PRELUDE_SCRIPTS) $(PREP)
  

  $(COMPILE_PRELUDE) $(PRELUDE_SCRIPTS) $@

golf_prelude.c: $(srcdir)/tool/compile_prelude.rb $(RBCONFIG) $(srcdir)/prelude.rb $(srcdir)/golf_prelude.rb $(PREP)
diff --git c/gem_prelude.rb i/gem_prelude.rb
index 71f30bd..09b8f3a 100644
--- c/gem_prelude.rb
+++ i/gem_prelude.rb
@@ -1,4 +1,3 @@
-# depends on: array.rb dir.rb env.rb file.rb hash.rb module.rb regexp.rb

vim: filetype=ruby

NOTICE: Ruby is during initialization here.

@@ -13,7 +12,8 @@ if defined?(Gem) then
module Kernel

  def gem(gem_name, *version_requirements)

•  Gem.push_gem_version_on_load_path(gem_name, *version_requirements)
  

•  Gem::QuickLoader.load_full_rubygems_library
  

•  gem gem_name, *version_requirements
  

  end
  private :gem
  end
  @@ -133,29 +133,27 @@ if defined?(Gem) then
  end

  module QuickLoader

•  @loaded_full_rubygems_library = false
  

•  def self.load_full_rubygems_library
  

•    return if @loaded_full_rubygems_library
  

•    return @loaded_full_rubygems_library if defined?(@loaded_full_rubygems_library)
  

•    @loaded_full_rubygems_library = true
  

•    @loaded_full_rubygems_library = false
  
     class << Gem
       undef_method(*Gem::GEM_PRELUDE_METHODS)
  

•      undef_method :const_missing
  

•      undef_method :method_missing
  

•    end
  

•    Kernel.module_eval do
  

•      undef_method :gem if method_defined? :gem
     end
  
     $".delete path_to_full_rubygems_library
     if $".any? {|path| path.end_with?('/rubygems.rb')}
       raise LoadError, "another rubygems is already loaded from #{path}"
     end
  

•    verbose, debug = $VERBOSE, $DEBUG
  

•    $VERBOSE = $DEBUG = nil
  

•    require 'rubygems'
  

•    @loaded_full_rubygems_library = true
  

•  ensure
  

•    $VERBOSE, $DEBUG = verbose, debug
   end
  
   def self.fake_rubygems_as_loaded
  

@@ -177,95 +175,6 @@ if defined?(Gem) then
end
end

•  GemPaths = {}
  

•  GemVersions = {}
  

•  def push_gem_version_on_load_path(gem_name, *version_requirements)
  

•    if version_requirements.empty?
  

•      unless GemPaths.has_key?(gem_name) then
  

•        raise Gem::LoadError, "Could not find RubyGem #{gem_name} (>= 0)\n"
  

•      end
  

•      # highest version gems already active
  

•      return false
  

•    else
  

•      if version_requirements.length > 1 then
  

•        QuickLoader.load_full_rubygems_library
  

•        return gem(gem_name, *version_requirements)
  

•      end
  

•      requirement, version = version_requirements[0].split
  

•      requirement.strip!
  

•      if loaded_version = GemVersions[gem_name] then
  

•        case requirement
  

•        when ">", ">=" then
  

•          return false if
  

•            (loaded_version <=> Gem.integers_for(version)) >= 0
  

•        when "~>" then
  

•          required_version = Gem.integers_for version
  

•          return false if loaded_version.first == required_version.first
  

•        end
  

•      end
  

•      QuickLoader.load_full_rubygems_library
  

•      gem gem_name, *version_requirements
  

•    end
  

•  end
  

•  def integers_for(gem_version)
  

•    numbers = gem_version.split(".").collect {|n| n.to_i}
  

•    numbers.pop while numbers.last == 0
  

•    numbers << 0 if numbers.empty?
  

•    numbers
  

•  end
  

•  def push_all_highest_version_gems_on_load_path
  

•    Gem.path.each do |path|
  

•      gems_directory = File.join(path, "gems")
  

•      if File.exist?(gems_directory) then
  

•        Dir.entries(gems_directory).each do |gem_directory_name|
  

•          next if gem_directory_name == "." || gem_directory_name == ".."
  

•          next unless gem_name = gem_directory_name[/(.*)-(.*)/, 1]
  

•          new_version = integers_for($2)
  

•          current_version = GemVersions[gem_name]
  

•          if !current_version or (current_version <=> new_version) < 0 then
  

•            GemVersions[gem_name] = new_version
  

•            GemPaths[gem_name] = File.join(gems_directory, gem_directory_name)
  

•          end
  

•        end
  

•      end
  

•    end
  

•    require_paths = []
  

•    GemPaths.each_value do |path|
  

•      if File.exist?(file = File.join(path, ".require_paths")) then
  

•        paths = File.read(file).split.map do |require_path|
  

•          File.join path, require_path
  

•        end
  

•        require_paths.concat paths
  

•      else
  

•        require_paths << file if File.exist?(file = File.join(path, "bin"))
  

•        require_paths << file if File.exist?(file = File.join(path, "lib"))
  

•      end
  

•    end
  

•    # "tag" the first require_path inserted into the $LOAD_PATH to enable
  

•    # indexing correctly with rubygems proper when it inserts an explicitly
  

•    # gem version
  

•    unless require_paths.empty? then
  

•      require_paths.first.instance_variable_set(:@gem_prelude_index, true)
  

•    end
  

•    # gem directories must come after -I and ENV['RUBYLIB']
  

•    $:[$:.index{|e|e.instance_variable_defined?(:@gem_prelude_index)}||-1,0] = require_paths
  

•  end
  

•  def const_missing(constant)
     QuickLoader.load_full_rubygems_library
  

@@ -285,10 +194,15 @@ if defined?(Gem) then

  extend QuickLoader

• def self.try_activate(path) # :nodoc:

•  QuickLoader.load_full_rubygems_library
  

• end

• end

  begin

• Gem.push_all_highest_version_gems_on_load_path

• require 'lib/rubygems/custom_require.rb'
• Gem::QuickLoader.fake_rubygems_as_loaded
  rescue Exception => e
  puts "Error loading gem paths on load path in gem_prelude"
  diff --git c/lib/rubygems.rb i/lib/rubygems.rb
  index d2214f6..29dd815 100644
  --- c/lib/rubygems.rb
  +++ i/lib/rubygems.rb
  @@ -1020,8 +1020,6 @@ end

module Kernel

• undef gem if respond_to? :gem # defined in gem_prelude.rb on 1.9

• Use Kernel#gem to activate a specific version of +gem_name+.

@@ -1098,13 +1096,34 @@ end

require 'rubygems/config_file'

+class << Gem

• verbose, debug = $VERBOSE, $DEBUG
• $VERBOSE = $DEBUG = nil

• Called from the custom_require to attempt to activate +path+

• Internal use only.

• def try_activate(path) # :doc:
• spec = Gem.searcher.find(path)
• return false unless spec
• Gem.activate(spec.name, "= #{spec.version}")
• return true
• end

+ensure

• $VERBOSE, $DEBUG = verbose, debug
  +end

Enables the require hook for RubyGems.

Ruby 1.9 allows --disable-gems, so we require it when we didn't detect a Gem

constant at rubygems.rb load time.

-require 'rubygems/custom_require' if gem_disabled or RUBY_VERSION < '1.9'
+require 'rubygems/custom_require'

Gem.clear_paths

diff --git c/lib/rubygems/custom_require.rb i/lib/rubygems/custom_require.rb
index 43b3136..343ad31 100644
--- c/lib/rubygems/custom_require.rb
+++ i/lib/rubygems/custom_require.rb
@@ -4,8 +4,6 @@

See LICENSE.txt for permissions.

#++

-require 'rubygems'

module Kernel

##

@@ -31,8 +29,7 @@ module Kernel
gem_original_require path
rescue LoadError => load_error
if load_error.message.end_with?(path) and

•   spec = Gem.searcher.find(path) then
  

•  Gem.activate(spec.name, "= #{spec.version}")
  

•   Gem.try_activate(path) then
   gem_original_require path
  

  else
  raise load_error
  @@ -42,5 +39,5 @@ module Kernel
  private :require
  private :gem_original_require

-end
+end unless Kernel.private_method_defined?(:gem_original_require)

--
Nobu Nakada

=end

[drbrain (Eric Hodel)]

=begin
On May 10, 2010, at 12:01, Evan Phoenix wrote:

Can we please get this committed for 1.9.2?

For what it's worth, I think Nobu has a better idea of whether this patch is acceptable or not than I do.

gem_prelude.rb is about how Ruby and RubyGems should interact so I don't think I am qualified to judge it.

=end

[mame (Yusuke Endoh)]

• Assignee set to nobu (Nobuyoshi Nakada)

=begin
Hi, Nobu

As Eric said, please judge your patch by yourself :-)

--
Yusuke Endoh [e-mail]
=end

[usa (Usaku NAKAMURA)]

• Status changed from Open to Assigned

=begin

=end

[nobu (Nobuyoshi Nakada)]

=begin
Tests are welcome.
=end

[nobu (Nobuyoshi Nakada)]

• Status changed from Assigned to Closed
• % Done changed from 0 to 100

=begin
This issue was solved with changeset r28200.
Aaron, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

=end

[runpaint (Run Paint Run Run)]

=begin
I think this caused some oddness. After building trunk, irb hung on load, seemingly looping indefinitely. The problem was my ~/.irbrc required a gem which irb could no longer find. I've yet to establish the reason for its amnesia, however, I am able to reproduce the hang by requiring a non-existent gem on startup:

 run@paint:/var/repos/mir/ruby →  ./ruby -d -W2 -v -rerrant -e 'p 1'
 ruby 1.9.3dev (2010-06-08 trunk 28202) [i686-linux]
 Exception `LoadError' at <internal:lib/rubygems/custom_require>:29 - cannot load such file -- errant
 Exception `LoadError' at <internal:lib/rubygems/custom_require>:29 - cannot load such file -- errant

The first exception is printed immediately. The second, 30 seconds or so later. Then the process just hangs indefinitely.
=end

[naruse (Yui NARUSE)]

• Status changed from Closed to Open

=begin

=end

[naruse (Yui NARUSE)]

• Target version changed from 1.9.2 to 2.0.0

=begin

=end

[wycats (Yehuda Katz)]

=begin
This bug is absolutely critical, and pushing it back from 1.9.2 to 1.9.x will make 1.9.2 unusable. It is trivial for this bug to impact Rails as well. In my opinion, while this bug exists Ruby 1.9.2 is broken and I cannot recommend its use.
=end

[tenderlovemaking (Aaron Patterson)]

=begin
I will try out nobu's patch and investigate why Run Paint was getting the failures.
=end

[rogerdpack (Roger Pack)]

=begin
Unfortunately this patch basically invalidates the usefulness of gem_prelude for ruby's gem generated binary scripts, since they all include a

gem 'xxx'

in them. But I suppose it's worth it to improve compatibility for people who have newer versions of a gem installed than the ones they actually use
=end

[tenderlovemaking (Aaron Patterson)]

• File undo-prelude-paths.diff undo-prelude-paths.diff added

=begin
How about an alternative solution to this problem? If someone does:

 require 'somegem'

The latest version of 'somegem' gets required. If someone does this:

 require 'rubygems'
 require 'somegem'

Then rubygems rules apply to subsequent gem requires. The motivation behind this behavior comes from the idea that if someone doesn't require rubygems, then they don't want or care about rubygems version and activation rules. If the user does require rubygems, then we assume that they care about such things.

Attached is a patch that adds this behavior (originally written by Evan Phoenix).
=end

[mame (Yusuke Endoh)]

=begin
Hi,

2010/7/3 Aaron Patterson [e-mail]:

How about an alternative solution to this problem? If someone does:

require 'somegem'

The latest version of 'somegem' gets required. If someone does this:

require 'rubygems'
require 'somegem'

Then rubygems rules apply to subsequent gem requires. The motivation behind this behavior comes from the idea that if someone doesn't require rubygems, then they don't want or care about rubygems version and activation rules. If the user does require rubygems, then we assume that they care about such things.

This may be a new feature. However, I don't object it because this
issue is actually a bug and we have no other solution (There will be
another solution theretically, but not in practice in the sense that
no one write a correct patch).

But,

Attached is a patch that adds this behavior (originally written by Evan Phoenix).

Your patch is not correct, unfortunately. make test-all does not run:

$ make test-all
./miniruby -I./lib -I.ext/common -I./- -r./ext/purelib.rb
./tool/runruby.rb --extout=.ext -- "./test/runner.rb"
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:103: warning: already
initialized constant VERSION
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:103: warning: already
initialized constant RubyGemsVersion
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:147: warning: already
initialized constant MUTEX
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:149: warning: already
initialized constant RubyGemsPackageVersion
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:155: warning: already
initialized constant WIN_PATTERNS
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:1018: warning: already
initialized constant MARSHAL_SPEC_DIR
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:1023: warning: already
initialized constant YAML_SPEC_DIR
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:613:in path': undefined method default_path' for Gem:Module (NoMethodError)
from /home/mame/work/ruby_1_9_2/lib/rubygems/source_index.rb:68:in
installed_spec_directories' from /home/mame/work/ruby_1_9_2/lib/rubygems/source_index.rb:58:in from_installed_gems'
from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:861:in source_index' from /home/mame/work/ruby_1_9_2/lib/rubygems/gem_path_searcher.rb:81:in init_gemspecs'
from /home/mame/work/ruby_1_9_2/lib/rubygems/gem_path_searcher.rb:13:in
initialize' from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:819:in new'
from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:819:in block in searcher' from <internal:prelude>:10:in synchronize'
from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:818:in searcher' from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:479:in find_files'
from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:961:in load_plugins' from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:1118:in <top
(required)>'
from /home/mame/work/ruby_1_9_2/lib/rubygems/custom_require.rb:31:in
require' from /home/mame/work/ruby_1_9_2/lib/rubygems/custom_require.rb:31:in require'
from internal:gem_prelude:158:in load_full_rubygems_library' from /home/mame/work/ruby_1_9_2/test/rubygems/gemutilities.rb:6:in <top (required)>'
from /home/mame/work/ruby_1_9_2/test/rubygems/test_gem_dependency_installer.rb:1:in
require_relative' from /home/mame/work/ruby_1_9_2/test/rubygems/test_gem_dependency_installer.rb:1:in <top (required)>'
from /home/mame/work/ruby_1_9_2/lib/rubygems/custom_require.rb:31:in
require' from /home/mame/work/ruby_1_9_2/lib/rubygems/custom_require.rb:31:in require'
from /home/mame/work/ruby_1_9_2/lib/test/unit.rb:55:in block in setup_argv' from /home/mame/work/ruby_1_9_2/lib/test/unit.rb:49:in each'
from /home/mame/work/ruby_1_9_2/lib/test/unit.rb:49:in setup_argv' from ./test/runner.rb:9:in '
make: *** [yes-test-all] Error 1

--
Yusuke Endoh [e-mail]

=end

[wycats (Yehuda Katz)]

=begin
We are about to ship a version of Ruby with a built in package manager with
the following property:

Given a package X with dependency Y, attempting to load X might require
dependency Z without any warning.

There is literally no other distribution of anything that would not consider
that property a major show-stopper. I am baffled about how this bug has
existed in the tracker so long, is considered "normal" priority, and has now
been bumped to 1.9.3 at the earliest.

Yehuda Katz
Architect | Engine Yard
(ph) 718.877.1325

On Fri, Jul 2, 2010 at 1:00 PM, Yusuke ENDOH [e-mail] wrote:

Hi,

2010/7/3 Aaron Patterson [e-mail]:

How about an alternative solution to this problem? If someone does:

require 'somegem'

The latest version of 'somegem' gets required. If someone does this:

require 'rubygems'
require 'somegem'

Then rubygems rules apply to subsequent gem requires. The motivation
behind this behavior comes from the idea that if someone doesn't require
rubygems, then they don't want or care about rubygems version and activation
rules. If the user does require rubygems, then we assume that they care
about such things.

This may be a new feature. However, I don't object it because this
issue is actually a bug and we have no other solution (There will be
another solution theretically, but not in practice in the sense that
no one write a correct patch).

But,

Attached is a patch that adds this behavior (originally written by Evan
Phoenix).

Your patch is not correct, unfortunately. make test-all does not run:

$ make test-all
./miniruby -I./lib -I.ext/common -I./- -r./ext/purelib.rb
./tool/runruby.rb --extout=.ext -- "./test/runner.rb"
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:103: warning: already
initialized constant VERSION
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:103: warning: already
initialized constant RubyGemsVersion
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:147: warning: already
initialized constant MUTEX
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:149: warning: already
initialized constant RubyGemsPackageVersion
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:155: warning: already
initialized constant WIN_PATTERNS
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:1018: warning: already
initialized constant MARSHAL_SPEC_DIR
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:1023: warning: already
initialized constant YAML_SPEC_DIR
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:613:in path': undefined method default_path' for Gem:Module (NoMethodError)
from /home/mame/work/ruby_1_9_2/lib/rubygems/source_index.rb:68:in
installed_spec_directories' from /home/mame/work/ruby_1_9_2/lib/rubygems/source_index.rb:58:in from_installed_gems'
from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:861:in
source_index' from /home/mame/work/ruby_1_9_2/lib/rubygems/gem_path_searcher.rb:81:in init_gemspecs'
from
/home/mame/work/ruby_1_9_2/lib/rubygems/gem_path_searcher.rb:13:in
initialize' from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:819:in new'
from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:819:in block in searcher' from <internal:prelude>:10:in synchronize'
from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:818:in searcher' from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:479:in find_files'
from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:961:in
load_plugins' from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:1118:in <top
(required)>'
from /home/mame/work/ruby_1_9_2/lib/rubygems/custom_require.rb:31:in
require' from /home/mame/work/ruby_1_9_2/lib/rubygems/custom_require.rb:31:in require'
from internal:gem_prelude:158:in load_full_rubygems_library' from /home/mame/work/ruby_1_9_2/test/rubygems/gemutilities.rb:6:in <top (required)>'
from
/home/mame/work/ruby_1_9_2/test/rubygems/test_gem_dependency_installer.rb:1:in
require_relative' from /home/mame/work/ruby_1_9_2/test/rubygems/test_gem_dependency_installer.rb:1:in <top (required)>'
from /home/mame/work/ruby_1_9_2/lib/rubygems/custom_require.rb:31:in
require' from /home/mame/work/ruby_1_9_2/lib/rubygems/custom_require.rb:31:in require'
from /home/mame/work/ruby_1_9_2/lib/test/unit.rb:55:in block in setup_argv' from /home/mame/work/ruby_1_9_2/lib/test/unit.rb:49:in each'
from /home/mame/work/ruby_1_9_2/lib/test/unit.rb:49:in setup_argv' from ./test/runner.rb:9:in '
make: *** [yes-test-all] Error 1

--
Yusuke Endoh [e-mail]

We are about to ship a version of Ruby with a built in package manager with the following property:

Given a package X with dependency Y, attempting to load X might require dependency Z without any warning.

There is literally no other distribution of anything that would not consider that property a major show-stopper. I am baffled about how this bug has existed in the tracker so long, is considered "normal" priority, and has now been bumped to 1.9.3 at the earliest.

Yehuda Katz
Architect | Engine Yard
(ph) 718.877.1325

On Fri, Jul 2, 2010 at 1:00 PM, Yusuke ENDOH <[e-mail]> wrote:

Hi,

2010/7/3 Aaron Patterson <[e-mail]>:

> How about an alternative solution to this problem?  If someone does:
>
>    require 'somegem'
>
> The latest version of 'somegem' gets required.  If someone does this:
>
>    require 'rubygems'
>    require 'somegem'
>
> Then rubygems rules apply to subsequent gem requires.  The motivation behind this behavior comes from the idea that if someone doesn't require rubygems, then they don't want or care about rubygems version and activation rules.  If the user *does* require rubygems, then we assume that they care about such things.

This may be a new feature.  However, I don't object it because this
issue is actually a bug and we have no other solution (There will be
another solution theretically, but not in practice in the sense that
no one write a correct patch).

But,

> Attached is a patch that adds this behavior (originally written by Evan Phoenix).

Your patch is not correct, unfortunately.  make test-all does not run:


$ make test-all
./miniruby -I./lib -I.ext/common -I./- -r./ext/purelib.rb
./tool/runruby.rb --extout=.ext  -- "./test/runner.rb"
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:103: warning: already
initialized constant VERSION
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:103: warning: already
initialized constant RubyGemsVersion
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:147: warning: already
initialized constant MUTEX
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:149: warning: already
initialized constant RubyGemsPackageVersion
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:155: warning: already
initialized constant WIN_PATTERNS
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:1018: warning: already
initialized constant MARSHAL_SPEC_DIR
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:1023: warning: already
initialized constant YAML_SPEC_DIR
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:613:in `path': undefined
method `default_path' for Gem:Module (NoMethodError)
       from /home/mame/work/ruby_1_9_2/lib/rubygems/source_index.rb:68:in
`installed_spec_directories'
       from /home/mame/work/ruby_1_9_2/lib/rubygems/source_index.rb:58:in
`from_installed_gems'
       from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:861:in `source_index'
       from /home/mame/work/ruby_1_9_2/lib/rubygems/gem_path_searcher.rb:81:in
`init_gemspecs'
       from /home/mame/work/ruby_1_9_2/lib/rubygems/gem_path_searcher.rb:13:in
`initialize'
       from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:819:in `new'
       from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:819:in `block
in searcher'
       from <internal:prelude>:10:in `synchronize'
       from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:818:in `searcher'
       from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:479:in `find_files'
       from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:961:in `load_plugins'
       from /home/mame/work/ruby_1_9_2/lib/rubygems.rb:1118:in `<top
(required)>'
       from /home/mame/work/ruby_1_9_2/lib/rubygems/custom_require.rb:31:in
`require'
       from /home/mame/work/ruby_1_9_2/lib/rubygems/custom_require.rb:31:in
`require'
       from <internal:gem_prelude>:158:in `load_full_rubygems_library'
       from /home/mame/work/ruby_1_9_2/test/rubygems/gemutilities.rb:6:in
`<top (required)>'
       from /home/mame/work/ruby_1_9_2/test/rubygems/test_gem_dependency_installer.rb:1:in
`require_relative'
       from /home/mame/work/ruby_1_9_2/test/rubygems/test_gem_dependency_installer.rb:1:in
`<top (required)>'
       from /home/mame/work/ruby_1_9_2/lib/rubygems/custom_require.rb:31:in
`require'
       from /home/mame/work/ruby_1_9_2/lib/rubygems/custom_require.rb:31:in
`require'
       from /home/mame/work/ruby_1_9_2/lib/test/unit.rb:55:in `block
in setup_argv'
       from /home/mame/work/ruby_1_9_2/lib/test/unit.rb:49:in `each'
       from /home/mame/work/ruby_1_9_2/lib/test/unit.rb:49:in `setup_argv'
       from ./test/runner.rb:9:in `<main>'
make: *** [yes-test-all] Error 1

--
Yusuke Endoh <[e-mail]>

=end

[wycats (Yehuda Katz)]

=begin
Here is a real-world demonstration of this bug:

~/Code/tmp /master > gem install actionpack -v 3.0.0.beta4
Successfully installed actionpack-3.0.0.beta4
1 gem installed
~/Code/tmp /master > gem install rack
Successfully installed rack-1.2.1
1 gem installed
~/Code/tmp /master > gem dependency actionpack -v 3.0.0.beta4
Gem actionpack-3.0.0.beta4
activemodel (= 3.0.0.beta4, runtime)
activesupport (= 3.0.0.beta4, runtime)
builder (~> 2.1.2, runtime)
erubis (~> 2.6.5, runtime)
i18n (~> 0.4.1, runtime)
rack (~> 1.1.0, runtime)
rack-mount (~> 0.6.3, runtime)
rack-test (~> 0.5.4, runtime)
tzinfo (~> 0.3.16, runtime)

~/Code/tmp /master > irb
ruby-1.9.2-head > require "action_dispatch"
=> true
ruby-1.9.2-head > Rack.release
=> "1.2"

=end

[luislavena (Luis Lavena)]

=begin
On Fri, Jul 2, 2010 at 6:19 PM, Yehuda Katz [e-mail] wrote:

We are about to ship a version of Ruby with a built in package manager with
the following property:

Given a package X with dependency Y, attempting to load X might require
dependency Z without any warning.

There is literally no other distribution of anything that would not consider
that property a major show-stopper. I am baffled about how this bug has
existed in the tracker so long, is considered "normal" priority, and has now
been bumped to 1.9.3 at the earliest.

While I agree with Yehuda that this is a bug that is high priority and
should be addressed in 1.9.2, I don't agree on the solutions.

I still wonder what is the reason of gem_prelude to exist, and if just
to avoid require 'rubygems', why not document the usage of
RUBYOPT=rubygems and get over it?

gem_prelude has not provided any value at all and instead added lot of
magic to the require process.

Not only that, but arbitrarily polluting my $LOAD_PATH and slowing
down any simple require statement due necessary File.stat performed in
all the elements of $LOAD_PATH before loading anything.

I wish there is a configure option like --disable-gem-prelude so I can
happily ship binaries for RubyInstaller without dealing with weird
errors reports I'm receiving in my personal inbox.

Thank you.

Luis Lavena
AREA 17

Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exupéry

=end

[tenderlovemaking (Aaron Patterson)]

=begin
On Sat, Jul 03, 2010 at 07:48:04AM +0900, Luis Lavena wrote:

On Fri, Jul 2, 2010 at 6:19 PM, Yehuda Katz [e-mail] wrote:

We are about to ship a version of Ruby with a built in package manager with
the following property:

Given a package X with dependency Y, attempting to load X might require
dependency Z without any warning.

There is literally no other distribution of anything that would not consider
that property a major show-stopper. I am baffled about how this bug has
existed in the tracker so long, is considered "normal" priority, and has now
been bumped to 1.9.3 at the earliest.

While I agree with Yehuda that this is a bug that is high priority and
should be addressed in 1.9.2, I don't agree on the solutions.

I still wonder what is the reason of gem_prelude to exist, and if just
to avoid require 'rubygems', why not document the usage of
RUBYOPT=rubygems and get over it?

gem_prelude has not provided any value at all and instead added lot of
magic to the require process.

I have to agree. I'm still trying to figure out the utility of
gem_prelude while sifting through code and attempting to resolve this
issue.

Why do we have gem prelude?

--
Aaron Patterson
http://tenderlovemaking.com/

Attachment: (unnamed)
=end

[evanphx (Evan Phoenix)]

• File undo-prelude-loadpath-2.diff undo-prelude-loadpath-2.diff added

=begin
I've updated the patch to pass test-all. There is a rubygems test for this bug, but it can't run properly because test-all runs under miniruby, which has no prelude, so I've disabled the test for now.

This patch runs test-all fine, and demonstrates the fact that using "require 'rubygems'" restores the system to 1.8 behaivor, therefore avoiding this bug.
=end

[shyouhei (Shyouhei Urabe)]

• Target version changed from 2.0.0 to 1.9.2

=begin
Can we see this a release blocker for 1.9.2?
=end

[mame (Yusuke Endoh)]

=begin
Hi,

2010/7/3 Evan Phoenix [e-mail]:

I've updated the patch to pass test-all. There is a rubygems test for this bug, but it can't run properly because test-all runs under miniruby, which has no prelude, so I've disabled the test for now.

This patch runs test-all fine, and demonstrates the fact that using "require 'rubygems'" restores the system to 1.8 behaivor, therefore avoiding this bug.

Great. I confirmed it indeed runs test-all.
But it emits some "already initialized constant" warnings.
It seems to allow to require "rubygems" twice. Is it ok?
If there is no objection, I'll commit this patch.

$ make test-all
./miniruby -I./lib -I.ext/common -I./- -r./ext/purelib.rb
./tool/runruby.rb --extout=.ext -- "./test/runner.rb"
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:108: warning: already
initialized constant VERSION
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:108: warning: already
initialized constant RubyGemsVersion
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:152: warning: already
initialized constant MUTEX
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:154: warning: already
initialized constant RubyGemsPackageVersion
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:160: warning: already
initialized constant WIN_PATTERNS
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:1023: warning: already
initialized constant MARSHAL_SPEC_DIR
/home/mame/work/ruby_1_9_2/lib/rubygems.rb:1028: warning: already
initialized constant YAML_SPEC_DIR
Loaded suite ./test/runner
Started

I'm now happy that everyone is tackling this issue.
I'd be happier if you had done so before rc1 is released :-)

--
Yusuke Endoh [e-mail]

=end

[runpaint (Run Paint Run Run)]

=begin
Evan's patch is fine by me. It's worth stating explicitly that this fixes the scenario described by wycats in comment 17.
=end

[rogerdpack (Roger Pack)]

=begin

This patch runs test-all fine, and demonstrates the fact that using "require 'rubygems'" > restores the system to 1.8 behaivor, therefore avoiding this bug.

Don't we want to restore the system default behavior only when the user uses a gem 'xxx' command? That's the case in question, is it not?
-r
=end

[wycats (Yehuda Katz)]

=begin
@runpaint (Run Paint Run Run) I don't think it does, since comment 17 does a simple require without requiring rubygems or using the #gem command.
=end

[raggi (James Tucker)]

=begin

On 6 Jul 2010, at 08:22, Roger Pack wrote:

I have to agree. I'm still trying to figure out the utility of
gem_prelude while sifting through code and attempting to resolve this
issue.

Why do we have gem prelude?

Because it allows us to avoid loading full rubygems, which takes quite
awhile on windows and jruby, and in general loads a lot of unnecessary
functionality [1]

I'm working on fixing this, if I could be allowed a little more time to generate a plugin load index and cut down the size of the spec cache for the sake of activation, then I could even remove the hack required to get this down to pretty much prelude speed (skipping plugin loads). As it is, I've got rubygems/fast.rb in the following branch within the same order of magnitude as gem_prelude, and doesn't break any of rubygems functionality apart from plugins (which are also violated by prelude). Furthermore this option will respect 3rd tier package manager patches, which is important if we're ever going to recover compatibility and communication folks like the Debian maintainers (something which I am working on too).

 http://github.com/rubygems/rubygems/compare/master...perflude

This patch set so far also massively reduces the immediately loaded file list:

 raggi@mbk: ~/dev/ext/rubygems % ruby -Ilib -r rubygems/fast -e 'puts $"' | wc -l
        7

Although this may not be "as light" as prelude (it is really damn close), it is semantically correct and should avoid the untold issues that users are already having due to prelude, and largely misunderstanding.

=end

[raggi (James Tucker)]

=begin

On 6 Jul 2010, at 10:14, James Tucker wrote:

On 6 Jul 2010, at 08:22, Roger Pack wrote:

I have to agree. I'm still trying to figure out the utility of
gem_prelude while sifting through code and attempting to resolve this
issue.

Why do we have gem prelude?

Because it allows us to avoid loading full rubygems, which takes quite
awhile on windows and jruby, and in general loads a lot of unnecessary
functionality [1]

I'm working on fixing this, if I could be allowed a little more time to generate a plugin load index and cut down the size of the spec cache for the sake of activation, then I could even remove the hack required to get this down to pretty much prelude speed (skipping plugin loads). As it is, I've got rubygems/fast.rb in the following branch within the same order of magnitude as gem_prelude, and doesn't break any of rubygems functionality apart from plugins (which are also violated by prelude). Furthermore this option will respect 3rd tier package manager patches, which is important if we're ever going to recover compatibility and communication folks like the Debian maintainers (something which I am working on too).

http://github.com/rubygems/rubygems/compare/master...perflude

This patch set so far also massively reduces the immediately loaded file list:

raggi@mbk: ~/dev/ext/rubygems % ruby -Ilib -r rubygems/fast -e 'puts $"' | wc -l
7

Although this may not be "as light" as prelude (it is really damn close), it is semantically correct and should avoid the untold issues that users are already having due to prelude, and largely misunderstanding.

I forgot to include simple benchmark examples:

http://gist.github.com/463942

=end

[runpaint (Run Paint Run Run)]

=begin

@runpaint (Run Paint Run Run) I don't think it does, since comment 17 does a simple require without requiring
rubygems or using the #gem command.

I misspoke. I meant that if it is preceded with require 'rubygems' it works as it should. IOW, the 1.8, and arguably correct, behaviour is available by explicitly requiring 'rubygems', meaning that scripts that work under 1.8 will behave in the same fashion under 1.9. Under 1.9, fans of brevity can omit require 'rubygems' if willing to accept the altered semantics. This is an uneasy compromise from any angle, but does at least retain backward compatibility.
=end

[raggi (James Tucker)]

=begin

On 6 Jul 2010, at 10:56, Run Paint Run Run wrote:

Issue #3140 has been updated by Run Paint Run Run.

@runpaint (Run Paint Run Run) I don't think it does, since comment 17 does a simple require without requiring
rubygems or using the #gem command.

I misspoke. I meant that if it is preceded with require 'rubygems' it works as it should. IOW, the 1.8, and arguably correct, behaviour is available by explicitly requiring 'rubygems', meaning that scripts that work under 1.8 will behave in the same fashion under 1.9. Under 1.9, fans of brevity can omit require 'rubygems' if willing to accept the altered semantics. This is an uneasy compromise from any angle, but does at least retain backward compatibility.

I don't agree, this is forcing people to add require 'rubygems' to their code, which is unacceptable as well. Introducing broken semantics into the core language will add support load to rubygems, bundler, debian, rails, and a bunch of other places. This is totally unfair in light of it basically being a poor approach at optimisation-by-replacement rather than optimisation by profiling and fixing the core problems. See my optimisations of rubygems (which at the moment the most significant impact is a hack to allow not loading plugins). There is more than can be done in this direction, and in my opinion gem_prelude should never have been created instead of doing these optimisations on core. To completely violate semantics and pass issues frequently down to users, and to suggest that they should then fix these issues by invoking bad practices is totally ridiculous, I really can't bring myself to respond to this kind of comment more lightly.

The situation is unacceptable, and its continuation is irresponsible.

---

http://redmine.ruby-lang.org/issues/show/3140

---

http://redmine.ruby-lang.org

=end

[runpaint (Run Paint Run Run)]

=begin

I don't agree, this is forcing people to add require 'rubygems' to their code,
which is unacceptable as well. Introducing broken semantics into the core
language will add support load to rubygems, bundler, debian, rails, and a
bunch of other places.

We're talking at cross purposes. Prior to 1.9, people already had to add require 'rubygems' to their code before requiring a gem. Therefore, given that 1.9.2 will be the first viable 1.9 release, the compulsion is precisely the same as before: on 1.8 and 1.9.2 users will be "forced" to require 'rubygems' before loading a gem under traditional semantics. Nothing has changed in this regard.

There is more than can be done in this direction... [deletia]

Which is also what I said. Nobody is suggesting that the current patch is anything approaching ideal. If your proposal meets the myriad requirements, then it will indeed be a welcome replacement.
=end

[raggi (James Tucker)]

=begin

On 6 Jul 2010, at 11:38, Run Paint Run Run wrote:

Issue #3140 has been updated by Run Paint Run Run.

I don't agree, this is forcing people to add require 'rubygems' to their code,
which is unacceptable as well. Introducing broken semantics into the core
language will add support load to rubygems, bundler, debian, rails, and a
bunch of other places.

We're talking at cross purposes. Prior to 1.9, people already had to add require 'rubygems' to their code before requiring a gem. Therefore, given that 1.9.2 will be the first viable 1.9 release, the compulsion is precisely the same as before: on 1.8 and 1.9.2 users will be "forced" to require 'rubygems' before loading a gem under traditional semantics. Nothing has changed in this regard.

But users are no longer forced to require rubygems, instead they will hit bugs, and then, due to the nature of most users, they will add those lines to their code, when in reality, it would be better if they use $RUBYOPT or the like. This means that pain is added to their workflow, that is all. It also means (and I'll put good money on this) that a lot of people will backlash and do exactly as you suggest, and actually add require 'rubygems' to their runtime code, which is a bad practice.

There is more than can be done in this direction... [deletia]

Which is also what I said. Nobody is suggesting that the current patch is anything approaching ideal. If your proposal meets the myriad requirements, then it will indeed be a welcome replacement.

Can someone summarise those requirements so I can make sure I don't miss any?

=end

[tenderlovemaking (Aaron Patterson)]

=begin
On Tue, Jul 06, 2010 at 04:22:22PM +0900, Roger Pack wrote:

I have to agree.  I'm still trying to figure out the utility of
gem_prelude while sifting through code and attempting to resolve this
issue.

Why do we have gem prelude?

Because it allows us to avoid loading full rubygems, which takes quite
awhile on windows and jruby, and in general loads a lot of unnecessary
functionality [1]

Why can't we opt-in to a faster rubygems, rather than having one shoved
down our throats? I'd rather see gem_prelude.rb moved to ext and opt in
to using it.

--
Aaron Patterson
http://tenderlovemaking.com/

Attachment: (unnamed)
=end

[mame (Yusuke Endoh)]

=begin
Hi,

I set a deadline of this discussion: 7/10 (Sat) 23:59 JST.
1.9.2 will be released in status quo unless we can agree on concrete
solution and write a patch.

IMO, it is no use just to complain. It is too late to broaden the
discussion. Let's focus on the issue at hand, and please write a
patch yourself.

Or, complain directly to the maintainer for rubygems.

--
Yusuke Endoh [e-mail]
=end

[raggi (James Tucker)]

=begin

On 6 Jul 2010, at 16:49, Yusuke Endoh wrote:

Issue #3140 has been updated by Yusuke Endoh.

Hi,

I set a deadline of this discussion: 7/10 (Sat) 23:59 JST.
1.9.2 will be released in status quo unless we can agree on concrete
solution and write a patch.

IMO, it is no use just to complain. It is too late to broaden the
discussion. Let's focus on the issue at hand, and please write a
patch yourself.

I have. http://github.com/rubygems/rubygems/compare/master...perflude

Or, complain directly to the maintainer for rubygems.

I am.

Is the above patch to rubygems actually acceptable for inclusion (via rubygems/fast.rb) to be included into ruby as a replacement for gem_prelude?

--
Yusuke Endoh [e-mail]

http://redmine.ruby-lang.org/issues/show/3140

---

http://redmine.ruby-lang.org

=end

[tenderlovemaking (Aaron Patterson)]

=begin
On Wed, Jul 07, 2010 at 12:49:35AM +0900, Yusuke Endoh wrote:

Issue #3140 has been updated by Yusuke Endoh.

Hi,

I set a deadline of this discussion: 7/10 (Sat) 23:59 JST.
1.9.2 will be released in status quo unless we can agree on concrete
solution and write a patch.

IMO, it is no use just to complain. It is too late to broaden the
discussion. Let's focus on the issue at hand, and please write a
patch yourself.

You're right. I think Evan's patch is a suitable fix for the release.

Or, complain directly to the maintainer for rubygems.

I complain to him every day. ;-)

Unfortunately he does not maintain gem prelude.

--
Aaron Patterson
http://tenderlovemaking.com/

Attachment: (unnamed)
=end

[evanphx (Evan Phoenix)]

=begin

On Jul 6, 2010, at 9:10 AM, James Tucker wrote:

On 6 Jul 2010, at 16:49, Yusuke Endoh wrote:

Issue #3140 has been updated by Yusuke Endoh.

Hi,

I set a deadline of this discussion: 7/10 (Sat) 23:59 JST.
1.9.2 will be released in status quo unless we can agree on concrete
solution and write a patch.

IMO, it is no use just to complain. It is too late to broaden the
discussion. Let's focus on the issue at hand, and please write a
patch yourself.

I have. http://github.com/rubygems/rubygems/compare/master...perflude

Or, complain directly to the maintainer for rubygems.

I am.

Is the above patch to rubygems actually acceptable for inclusion (via rubygems/fast.rb) to be included into ruby as a replacement for gem_prelude?

No, the patch isn't acceptable as going into 1.9.2, because it has not gone through the normal rigor of rubygems testing. I see a few issues with it straight away.

• Evan

--
Yusuke Endoh [e-mail]

http://redmine.ruby-lang.org/issues/show/3140

---

http://redmine.ruby-lang.org

=end

[raggi (James Tucker)]

=begin

On 6 Jul 2010, at 17:12, Aaron Patterson wrote:

Or, complain directly to the maintainer for rubygems.

I complain to him every day. ;-)

Unfortunately he does not maintain gem prelude.

It seems like no one does, that's why I'm trying to find out what the aforementioned requirements list is.

I need that to make rubygems fulfil it all, in order to replace prelude with rubygems for real, which I think would be a preferable solution for everyone involved, no?

Is anyone able to please provide such a list?

=end

[raggi (James Tucker)]

=begin

On 6 Jul 2010, at 17:18, Evan Phoenix wrote:

On Jul 6, 2010, at 9:10 AM, James Tucker wrote:

On 6 Jul 2010, at 16:49, Yusuke Endoh wrote:

Issue #3140 has been updated by Yusuke Endoh.

Hi,

I set a deadline of this discussion: 7/10 (Sat) 23:59 JST.
1.9.2 will be released in status quo unless we can agree on concrete
solution and write a patch.

IMO, it is no use just to complain. It is too late to broaden the
discussion. Let's focus on the issue at hand, and please write a
patch yourself.

I have. http://github.com/rubygems/rubygems/compare/master...perflude

Or, complain directly to the maintainer for rubygems.

I am.

Is the above patch to rubygems actually acceptable for inclusion (via rubygems/fast.rb) to be included into ruby as a replacement for gem_prelude?

No, the patch isn't acceptable as going into 1.9.2, because it has not gone through the normal rigor of rubygems testing. I see a few issues with it straight away.

After discussion various of the social and release engineering issues with Evan, I want to provide a +1 for his patch for the 1.9.2 release.

There are still remaining bugs with gem_prelude.rb that concern me, but in the interest of progress I agree we should move forward and address these in a more organised discussion between the rubygems team and the ruby team by the next release in order to reduce user error reports for all involved, and our users.

=end

[mame (Yusuke Endoh)]

=begin
Hi,

2010/7/7 Evan Phoenix [e-mail]:

On Jul 6, 2010, at 8:49 AM, Yusuke Endoh wrote:

I set a deadline of this discussion: 7/10 (Sat) 23:59 JST.
1.9.2 will be released in status quo unless we can agree on concrete
solution and write a patch.

There are a number of people who have agreed my current patch is acceptable. Please consider it for 1.9.2. If there is more work that needs to be done on it, please let me know.

In [ruby-core:31031], I've already sent positive thoughts to your patch.

But I'm concerned a little that make test-all emits some warnings, as I
said in [ruby-core:31031]. Is it ok? If you say ok, I'll commit your
patch.

Thanks for your contribution!

--
Yusuke Endoh [e-mail]

=end

[evanphx (Evan Phoenix)]

• File undo-prelude-loadpath-3.diff undo-prelude-loadpath-3.diff added

=begin
I've fixed the issue with the warnings from redefining some constants.

I'd like to get this patch committed to 1.9.2 and 1.9.3.

Thanks!

=end

[drbrain (Eric Hodel)]

=begin
On Jul 6, 2010, at 10:25, Evan Phoenix wrote:

There are a number of people who have agreed my current patch is acceptable. Please consider it for 1.9.2. If there is more work that needs to be done on it, please let me know.

Hi.

I would like for Evan to be the Official Maintainer of gem_prelude.

=end

[mame (Yusuke Endoh)]

=begin
Hi,

I've fixed the issue with the warnings from redefining some constants.

I'd like to get this patch committed to 1.9.2 and 1.9.3.

I've commited the patch to trunk.
I'll commit it to ruby_1_9_2 tomorrow unless any critical problem
is reported (such as build trouble on windows).

FYI: rc2 will be released in the middle of this month.

Thanks,

--
Yusuke Endoh [e-mail]
=end

[mame (Yusuke Endoh)]

• Status changed from Open to Closed

=begin
This issue was solved with changeset r28570.
Aaron, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

=end

[luislavena (Luis Lavena)]

=begin
On Wed, Jul 7, 2010 at 2:57 PM, Roger Pack [e-mail] wrote:

I've commited the patch to trunk.
I'll commit it to ruby_1_9_2 tomorrow unless any critical problem
is reported (such as build trouble on windows).

Sorry to throw a wrench in this one...
This used to work...

C:\dev\ruby\downloads>ruby -v
ruby 1.9.3dev (2010-07-08) [i386-mingw32]

C:\dev\ruby\downloads>gem search rdoc
C:/installs/ruby_trunk_installed/lib/ruby/gems/1.9.1/specifications/rdoc-data-2.5.1.gemspec:18:
[BUG] gc_sweep(): unknown data type 0x0(0x230bd19)
ruby 1.9.3dev (2010-07-08) [i386-mingw32]

What version of RubyGems? Is this based on RubyInstaller? "gem" will
load directly RubyGems, which is not the intend of prelude.

--
Luis Lavena
AREA 17

Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exupéry

=end

[mame (Yusuke Endoh)]

=begin
Hi,

2010/7/8 Roger Pack [e-mail]:

I've commited the patch to trunk.
I'll commit it to ruby_1_9_2 tomorrow unless any critical problem
is reported (such as build trouble on windows).

Sorry to throw a wrench in this one...
This used to work...

C:\dev\ruby\downloads>ruby -v
ruby 1.9.3dev (2010-07-08) [i386-mingw32]

C:\dev\ruby\downloads>gem search rdoc
C:/installs/ruby_trunk_installed/lib/ruby/gems/1.9.1/specifications/rdoc-data-2.5.1.gemspec:18:
[BUG] gc_sweep(): unknown data type 0x0(0x230bd19)
ruby 1.9.3dev (2010-07-08) [i386-mingw32]

An obvious critical problem was reported, so I didn't commit the
patch to ruby_1_9_2.

Because this patch consists of just Ruby-level fix, I guess that
another potential problem was stimulated or there was some sort
of mistake.
I heard from eban (not evan) that he re-tested this on mingw32
but didn't reproduce this problem.

Roger, could you re-try to build from scratch? Could you apply
the test to ruby_1_9_2 and check it is reproducible?

I'm really disappointed that no one seems to investigate this
problem (except Luis's [ruby-core:31110]). No one want this issue
to be fixed, right? Otherwise, please think that it is a problem
as yourself. At least, one who becomes a maintainer should show
readiness to work at times like this.
At this rate, 1.9.2 will be released with this issue left.

--
Yusuke Endoh [e-mail]

=end

[luislavena (Luis Lavena)]

=begin
On Sun, Jul 11, 2010 at 10:23 AM, Yusuke ENDOH [e-mail] wrote:

Hi,

2010/7/8 Roger Pack [e-mail]:

I've commited the patch to trunk.
I'll commit it to ruby_1_9_2 tomorrow unless any critical problem
is reported (such as build trouble on windows).

Sorry to throw a wrench in this one...
This used to work...

C:\dev\ruby\downloads>ruby -v
ruby 1.9.3dev (2010-07-08) [i386-mingw32]

C:\dev\ruby\downloads>gem search rdoc
C:/installs/ruby_trunk_installed/lib/ruby/gems/1.9.1/specifications/rdoc-data-2.5.1.gemspec:18:
[BUG] gc_sweep(): unknown data type 0x0(0x230bd19)
ruby 1.9.3dev (2010-07-08) [i386-mingw32]

An obvious critical problem was reported, so I didn't commit the
patch to ruby_1_9_2.

This means that Evan Phoenix's patch hasn't been applied?

Because this patch consists of just Ruby-level fix, I guess that
another potential problem was stimulated or there was some sort
of mistake.
I heard from eban (not evan) that he re-tested this on mingw32
but didn't reproduce this problem.

Roger, could you re-try to build from scratch?  Could you apply
the test to ruby_1_9_2 and check it is reproducible?

I'm investigating right now the report of Roger.

I'm really disappointed that no one seems to investigate this
problem (except Luis's [ruby-core:31110]).  No one want this issue
to be fixed, right?  Otherwise, please think that it is a problem
as yourself.  At least, one who becomes a maintainer should show
readiness to work at times like this.

I've shared in the past my concerns about this. RubyInstaller provides
a simple command base to run the whole compilation environment so test
for this on Windows can be easily achieved, but seems none of the
Ruby-Core Windows developers have pay attention to it.

Sometimes we have to deal with strange bug reports and try to
reproduce, which is hard, sometimes we have to deal with fluke
generated by 3rd party applications (as yesterday mkdir_p case)

At this rate, 1.9.2 will be released with this issue left.

If you give me a few hours, I can investigate this.

Thank you.

Luis Lavena
AREA 17

Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exupéry

=end

[luislavena (Luis Lavena)]

=begin
On Sun, Jul 11, 2010 at 10:23 AM, Yusuke ENDOH [e-mail] wrote:

Roger, could you re-try to build from scratch?  Could you apply
the test to ruby_1_9_2 and check it is reproducible?

C:\Users\Luis\Projects\oss\oci\rubyinstaller>ruby -v
ruby 1.9.2dev (2010-07-11 revision 28610) [i386-mingw32]

C:\Users\Luis\Projects\oss\oci\rubyinstaller>gem -v
1.3.7

C:\Users\Luis\Projects\oss\oci\rubyinstaller>gem search rdoc

*** LOCAL GEMS ***

rdoc (2.5.8)
rdoc-data (2.5.3)

Cannot reproduce with 1.9.2, testing right now with trunk and will post results.

Luis Lavena
AREA 17

Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exupéry

=end

[mame (Yusuke Endoh)]

=begin
Hi,

2010/7/11 Luis Lavena [e-mail]:

An obvious critical problem was reported, so I didn't commit the
patch to ruby_1_9_2.

This means that Evan Phoenix's patch hasn't been applied?

Yes, at the moment.

I'm really disappointed that no one seems to investigate this
problem (except Luis's [ruby-core:31110]). No one want this issue
to be fixed, right? Otherwise, please think that it is a problem
as yourself. At least, one who becomes a maintainer should show
readiness to work at times like this.

I've shared in the past my concerns about this. RubyInstaller provides
a simple command base to run the whole compilation environment so test
for this on Windows can be easily achieved, but seems none of the
Ruby-Core Windows developers have pay attention to it.

Some ruby-core windows developers build and test trunk every day.
I asked some of them to try to reproduce the issue, but none could.

In fact, almost all of them seem not to be interested in rubygems.
I hoped that beneficiaries work seriously about this.

I admit ruby-core developpers' status should be opened to a third
person (though, language barrier is difficult!), and I think Luis's
effort is really great.

At this rate, 1.9.2 will be released with this issue left.

If you give me a few hours, I can investigate this.

Unfortunately, rc2 was tagged at r28618 in the last hour.
My reminder might be also too late. Sorry.
Yugui, can we still make it?

--
Yusuke Endoh [e-mail]

=end

[mame (Yusuke Endoh)]

=begin
Uh-oh, rc2 has been released.
This issue entered a more difficult phase...

2010/7/12 Yusuke ENDOH [e-mail]:

If you give me a few hours, I can investigate this.

Unfortunately, rc2 was tagged at r28618 in the last hour.
My reminder might be also too late.  Sorry.
Yugui, can we still make it?

--
Yusuke Endoh [e-mail]

=end

[luislavena (Luis Lavena)]

=begin
On Sun, Jul 11, 2010 at 11:51 AM, Luis Lavena [e-mail] wrote:

Cannot reproduce with 1.9.2, testing right now with trunk and will post results.

C:\Users\Luis\Projects\oss\oci\rubyinstaller\ruby193-trunk>ruby -v
ruby 1.9.3dev (2010-07-11 trunk 28617) [i386-mingw32]

C:\Users\Luis\Projects\oss\oci\rubyinstaller\ruby193-trunk>gem -v
1.3.7

C:\Users\Luis\Projects\oss\oci\rubyinstaller\ruby193-trunk>gem search rdoc

*** LOCAL GEMS ***

rdoc (2.5.8)
rdoc-data (2.5.3)

Not reproducible with either 1.9.2 or 1.9.3 (trunk).

Luis Lavena
AREA 17

Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exupéry

=end

[luislavena (Luis Lavena)]

=begin
On Sun, Jul 11, 2010 at 12:26 PM, Yusuke ENDOH [e-mail] wrote:

Uh-oh, rc2 has been released.
This issue entered a more difficult phase...

This means that Evan Phoenix fix hasn't been integrated?

That is a terrible news, considering that Roger Pack's report was
highly unlikely be related or associated with Evan's fix.

Where we can see the schedule of releases? Aiming in the future not
miss those deadlines with critical things that could be problematic
for lot of developers.

Thank you.

Luis Lavena
AREA 17

Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exupéry

=end

[mame (Yusuke Endoh)]

=begin
Hi,

2010/7/12 Luis Lavena [e-mail]:

On Sun, Jul 11, 2010 at 12:26 PM, Yusuke ENDOH [e-mail] wrote:

Uh-oh, rc2 has been released.
This issue entered a more difficult phase...

This means that Evan Phoenix fix hasn't been integrated?

Yes.

That is a terrible news, considering that Roger Pack's report was
highly unlikely be related or associated with Evan's fix.

Agreed. But unless the problem is reported without the fix, we can't
help but be conservative to include the fix.

Where we can see the schedule of releases? Aiming in the future not
miss those deadlines with critical things that could be problematic
for lot of developers.

http://redmine.ruby-lang.org/wiki/ruby-19/ReleasePlan

Originally, rc1 had been the last chance to include the patch with
public preview. But Yugui planned to release rc2 to spare us some
time for some issues. We missed the true last chance.

I talked with Yugui a few minutes, and

• she had had no reluctance to include the fix if it is dependable
  (but I don't know whether she will accept the fix even after rc2)

• she didn't think this issue was serious because it could be
  worked around by using bundler (? I'm not sure that I understand
  her opinion precisely)

--
Yusuke Endoh [e-mail]

=end

[evanphx (Evan Phoenix)]

=begin

On Jul 11, 2010, at 9:13 AM, Yusuke ENDOH wrote:

Hi,

2010/7/12 Luis Lavena [e-mail]:

On Sun, Jul 11, 2010 at 12:26 PM, Yusuke ENDOH [e-mail] wrote:

Uh-oh, rc2 has been released.
This issue entered a more difficult phase...

This means that Evan Phoenix fix hasn't been integrated?

Yes.

That is a terrible news, considering that Roger Pack's report was
highly unlikely be related or associated with Evan's fix.

Agreed. But unless the problem is reported without the fix, we can't
help but be conservative to include the fix.

This is very bad news. This fix effects a lot of people. I did not respond because Roger indicate he could not reproduce it himself. No one has been able to reproduce the issue, including Roger himself. If that is the reason the fix did not make it into 1.9.2, that's very sad.

Where we can see the schedule of releases? Aiming in the future not
miss those deadlines with critical things that could be problematic
for lot of developers.

http://redmine.ruby-lang.org/wiki/ruby-19/ReleasePlan

Originally, rc1 had been the last chance to include the patch with
public preview. But Yugui planned to release rc2 to spare us some
time for some issues. We missed the true last chance.

I talked with Yugui a few minutes, and

• she had had no reluctance to include the fix if it is dependable
  (but I don't know whether she will accept the fix even after rc2)

• she didn't think this issue was serious because it could be
  worked around by using bundler (? I'm not sure that I understand
  her opinion precisely)

This is exactly backwards from what I understand. This bug means that bundler can not run on 1.9.2, and thus rails 3.0 can not run on 1.9.2. This is what Yehuda has told me.

If 1.9.2 can not support rails 3, I fear that the ruby community will reject 1.9.2 right away. I ask that matz, yugui, and all ruby-core developers consider this before allowing the bug to go unfixed.

The pace of this bug has frustrated everyone, please don't let it die now.

• Evan

--
Yusuke Endoh [e-mail]

=end

[nobu (Nobuyoshi Nakada)]

=begin
Hi,

At Mon, 12 Jul 2010 16:41:06 +0900,
Evan Phoenix wrote in [ruby-core:31206]:

This is very bad news. This fix effects a lot of people. I
did not respond because Roger indicate he could not reproduce
it himself. No one has been able to reproduce the issue,
including Roger himself. If that is the reason the fix did
not make it into 1.9.2, that's very sad.

Sorry, I can't find if Roger tried it with 1.9.2, not with
trunk. In [ruby-core:31108], he found it with trunk on mingw
but couldn't reproduce it on linux.

This is exactly backwards from what I understand. This bug
means that bundler can not run on 1.9.2, and thus rails 3.0
can not run on 1.9.2. This is what Yehuda has told me.

Bundler also depends on this behavior?

--
Nobu Nakada

=end

[wycats (Yehuda Katz)]

=begin
Bundler can work around this issue, but only using very brittle hacks that could easily break in a future patch level. Additionally, this would mean that the only reliable way to use Ruby 1.9.2 with Rubygems is bundler. I like bundler and all, but requiring it as a pre-requisite for using Ruby with Rubygems is unacceptable.

Bottom line: I cannot personally support a version of Ruby that can silently require a dependency that is explicitly disallowed by the parent. That possibility is ridiculous, and I'm amazed that, for months, it has been downplayed.

If the problem is that ruby-core doesn't use RubyGems that much, and isn't willing to avoid breaking fundamental package manager properties, perhaps it's worth reconsidering including it in Ruby 1.9.
=end

[matz (Yukihiro Matsumoto)]

=begin
Hi,

In message "Re: [ruby-core:31215] [Bug #3140] gem activation has changed between 1.8 and 1.9"
on Tue, 13 Jul 2010 02:51:04 +0900, Yehuda Katz [e-mail] writes:

|Bottom line: I cannot personally support a version of Ruby that can silently require a dependency that is explicitly disallowed by the parent. That possibility is ridiculous, and I'm amazed that, for months, it has been downplayed.
|
|If the problem is that ruby-core doesn't use RubyGems that much, and isn't willing to avoid breaking fundamental package manager properties, perhaps it's worth reconsidering including it in Ruby 1.9.

I am sorry that I haven't followed this issue. As far as I
understand, this is caused by unfortunate timing. Eric was busy at
the critical timing, most of us hesitated modifying it without Eric's
concent, and passing the maintainer role to Evan was late.

I think 1.9.2 should run Rails3. If I understand what Yehuda told me
correctly, this issue could be a show stopper. I know RC2 is so late
to add a change, but loosing Rails3 people is critical for 1.9.2
popularity. I don't want to see 1.8.6 tragedy again. It took us
years to overcome slight incompatibility with ActiveSupport. Don't be
bureaucratic.

						matz.

=end

[evanphx (Evan Phoenix)]

=begin

On Jul 12, 2010, at 1:25 PM, Roger Pack wrote:

Roger, could you re-try to build from scratch? Could you apply
the test to ruby_1_9_2 and check it is reproducible?

Ok rebuild it from scratch.
After installing these gems:
$ gem install redcar rdoc-data win32console rails

then the bug resurfaces.

For the adventurous:
http://myfavoritepal.com:/incoming/trunk3_installed.7z
contains the binary that fails.

I don't have access to a windows 7 machine to test this, nor do I have any knowledge of windows 7 debugging.

I've looked back over your original backtrace. Assuming your still seeing the same issue (gc_sweep(), unknown data), I'm guessing that my patch is not the core cause of the issue, but is rather just running some code path that causes a GC bug.

If this is the case, then the GC bug exists already and needs to be fixed whether or not my patch is applied.

If this is considered a show stopper for 1.9.2, then it's the GC bug that is a show stopper, not my patch. Not applying my patch because of the GC bug solves nothing, but the GC bug will still exist and users will hit it some other way.

I currently do not have the knowledge to track down the GC bug, but I hope that someone on ruby-core does and can assist.

• Evan Phoenix

It's only on one machine though....maybe somebody else with windows 7
could try it out and see if it fails? I haven't gotten it to fail on
Linux or XP.

I'll try with 1_9_2 next.

-r

=end

[raggi (James Tucker)]

=begin

On 12 Jul 2010, at 19:47, Yukihiro Matsumoto wrote:

Hi,

In message "Re: [ruby-core:31215] [Bug #3140] gem activation has changed between 1.8 and 1.9"
on Tue, 13 Jul 2010 02:51:04 +0900, Yehuda Katz [e-mail] writes:

|Bottom line: I cannot personally support a version of Ruby that can silently require a dependency that is explicitly disallowed by the parent. That possibility is ridiculous, and I'm amazed that, for months, it has been downplayed.
|
|If the problem is that ruby-core doesn't use RubyGems that much, and isn't willing to avoid breaking fundamental package manager properties, perhaps it's worth reconsidering including it in Ruby 1.9.

I am sorry that I haven't followed this issue. As far as I
understand, this is caused by unfortunate timing. Eric was busy at
the critical timing, most of us hesitated modifying it without Eric's
concent, and passing the maintainer role to Evan was late.

In a way I agree that the timing on this specific ticket has been unfortunate, but the issue in question here, and other issues have existed ever since gem_prelude.rb was integrated.

Evans patches add a single viable (hack) to workaround the issue, but they do not remove secondary gem_prelude.rb issues.

Just for the sake of making sure that there is awareness, I am going to detail some more experiences here:

6 months ago I was helping a friend of mine try to debug issues in an app written to perform some very simple serving of xml processed through xslt. Something ruby, rack, and thin/cgi are very capable of. The user eventually dropped using ruby for the project after she realised that the bug I tracked down I was only able to track down because I'm intimately familiar with all of these projects and with rubygems and the gem_prelude.rb issues. To her, as a non-rubyist this was totally unacceptable, and she subsequently rewrote what she needed in C, and will not be using ruby in the foreseeable future. This is not an uncommon story, most users wouldn't even get an answer as to the source of their issues. Arguably such a user should not have been using 1.9, however, that seemed like a good idea to her, due to the presentation in her OS package manager. Documentation immediately available did not direct her otherwise.

We have had numerous complaints in #rubygems regarding issues introduced by gem_prelude, and whilst I won't speak for the other committers, this does take away from support time provided to users with other problems, and reduces time I spend patching.

The same is true for Rack.

The same is true for EventMachine.

There's also a clobber that is absolutely horrible between older versions of 'aasm' and 'state_machine' that 1.9.x makes almost impossible for users to solve correctly at this time. This is just yet another example. I could provide a full audit of similar clobbers soon once I've finished updating my gem mirror and gauntlet code. Such a thing won't actually solve anything though.

If you 'gem install mongrel' on 1.9, it will fail (for a valid reason). The directory is not removed because that would prevent users from debugging the build process by reading 'gem_make.out' that is left there for this purpose. gem_prelude.rb is unaware of this, and will allow "require 'mongrel'" to pass, resulting in an exception that to end users is totally cryptic (it will fail on http11.so, which won't exist). The cause of this odd error, and indeed "require 'mongrel'" even trying to load is the fault of gem_prelude. I've seen bug reports to rubygems about this issue to, but we do not really have control over prelude (although I fixed some bugs in it a while back regarding ConfigMap, the sync had to be done by a person with commit on ruby-core, honestly, I don't know if that even happened, but it wasn't a blocker for me).

There is also of course the major bug that Yehuda has pointed out.

There are bugs also in any gem that uses other facets of the rubygems api, where gem_prelude is incomplete or lacking (the ConfigMap example which I fixed only being one example).

Whilst I appreciate that Eric has a very laid back attitude deferring to ruby-cores desires on the matter, I say as I know many others do here, that gem_prelude.rb is insufficient as a replacement for RubyGems. More than this, short of really embedding RubyGems, I really can't see all of the issues being solved with any level of completeness - to do so would fundamentally require that gem_prelude.rb passes both all of rubygems test cases, and in-the-wild use case coverage that we don't even have in tests. RubyGems is not "bloated" at what it does, honestly, take a look at the code base, it's short enough to really grok in a few hours. Pretending that 300 lines of code can replace all of its domain expertise is not really reasonable, IMO.

I don't mean to sound like RubyGems is the be-all and end-all of packaging, even for ruby, I know that it is not - however, a broken partial reimplementation has been proven in 1.9 to be dire for a great deal of projects, and there are plenty of in-the-wild examples of this. Those examples will only become more and more frequent when ruby 192 is touted as a "production ready release", and the overall impact could be quite disastrous in reality. People already vested in ruby will not move away, more than likely, but they will pass around hacks to solve the problems, or rvm will begin to carry unofficial patches, or many other possible workarounds. Newcomers or opposers to the language will simply have total fuel for their fire however. I'm not being sensationalist, this is already happening, and I'm not the only one to have seen it.

Here is another from Rack that I cleared today: http://github.com/rack/rack/issues/closed#issue/35/comment/308670 Of course this is just a repetition of the bug demonstrated by Yehuda that is critical for bundler and rails 3.

Yehuda has said that he cannot support a release that has this issue, I will go further as to say that personally, I will state to anyone that asks, that any release of ruby that still contains this abomination is specifically not production ready. I say this because it will continually add support load for at least three of the projects that I commit on, that cover between all of them, some of the well used ones in our community outside of (and including, by dependency) rails. The most effective way for me to get that time back is to push a culture that rejects versions that will still bring up these issues. This would be a purely preventative measure!

Yes some of the content of this mail is heartfelt. I trust that you will recognise that there is reason (read: pain) behind that. If it made this mail read badly, I can only apologise for that part.

I do hope that this provides some more of an idea of the issues behind this, and why some of us feel so strongly about it. It is also why I sent a tweet to you on the topic a few weeks ago.

Evan did ask me not to push this issue for 192, so that the release can happen, and I have conceded that the more "minor" issues (that are not blockers to rails) can be ignored /for this release/, however I am still deeply unhappy about this as there will still be support load induced by the inherent bugs in the gem_prelude.rb approach detailed above. That is the only reason I am writing this mail. I am still ok for 1.9.2 to go ahead in such a way that prelude may have some issues, provided it doesn't block rails (i.e. with Evans patch plus whatever is required for stability).

I would however like to see this resolved for 1.9.3, and I am working on patches to rubygems that should make inclusion of rubygems.rb viable for 1.9.3, rather than using the gem_prelude.rb hack. There is one requirement I know of - the 0 stack requirement - that I may not meet, but I think this will have to be conceded for the sake of the worlds sanity. This part should obviously be discussed elsewhere, for 1.9.3.

Apologies once again, if any of this comes across as a rant, but it's important that awareness is there, particularly that the rails issue is not the only one.

I think 1.9.2 should run Rails3. If I understand what Yehuda told me
correctly, this issue could be a show stopper. I know RC2 is so late
to add a change, but loosing Rails3 people is critical for 1.9.2
popularity. I don't want to see 1.8.6 tragedy again. It took us
years to overcome slight incompatibility with ActiveSupport. Don't be
bureaucratic.

 					matz.

=end

[raggi (James Tucker)]

=begin

On 13 Jul 2010, at 01:04, Roger Pack wrote:

If you 'gem install mongrel' on 1.9, it will fail (for a valid reason). The directory is not removed because that would prevent users from debugging the build process by reading 'gem_make.out' that is left there for this purpose.

Yeah I've run into this before. I even filed a bug for it with
rubygems, but nothing was done. Maybe Eric is too busy?

It is NOT a bug in rubygems. It's a bug in gem_prelude.rb. gem_prelude.rb is broken by design. I may have even marked that ticket as invalid for you.

This is the same as your bin_path 'enhancements' for prelude, they're making these issues worse.

=end

[mame (Yusuke Endoh)]

=begin
2010/7/13 Yehuda Katz [e-mail]:

If the problem is that ruby-core doesn't use RubyGems that much, and isn't willing to avoid breaking fundamental package manager properties, perhaps it's worth reconsidering including it in Ruby 1.9.

Agreed.

BTW, I don't see why you did not complain until rc1 is released.
I gave and noticed you a chance to test your product [ruby-core:30543].
In addition, the preview plan was announced in May [ruby-core:28665].

In fact, I heard from Shyouhei that you had noticed some trouble about
this issue in June (though I cannot know the detail because you did
not complain formally).

You actually had a chance. In this case, I can't help but say that you
are too uncooperative, or even opposing to 1.9.2 release. This result
is what you asked for.

In this case I guess we could manage this issue, thanks to Evan's patch.
But, please please cooperate us in next time.

--
Yusuke Endoh [e-mail]

=end

[mame (Yusuke Endoh)]

=begin
Hi,

2010/7/13 Yukihiro Matsumoto [e-mail]:

I am sorry that I haven't followed this issue. As far as I
understand, this is caused by unfortunate timing. Eric was busy at
the critical timing

Not only at that timing. He is always busy.

[ruby-core:28689] [ruby-core:28781] [ruby-core:28926]
[ruby-core:30127] [ruby-core:30129] [ruby-core:30130]
[ruby-core:30151] [ruby-core:30189] [ruby-core:30456]
[ruby-core:30619] [ruby-core:30621] [ruby-core:30792]
[ruby-core:30907]

Of course I can't blame his busyness, but looking objectively,
he looks like ineligible for a maintainer.

I think 1.9.2 should run Rails3.

I hope too. I wanted them to know the very poor maintenance status
of rubygems and to do it themselves, as they are beneficiaries.

OTOH there are also many people who wait for 1.9.2 but don't need
rubygems nor rails.

If I understand what Yehuda told me
correctly, this issue could be a show stopper.

I don't agree to see the issue as a show stopper unless some active
developer is assigned to the issue. Otherwise it would take forever,
literally.

In practice, I think Evan's patch is a meeting point, if Roger can
confirm the [BUG] does not occur on ruby_1_9_2 with Evan's patch.
If it does not occur on ruby_1_9_2, I also suspect it is due to lazy
sweep ([ruby-core:31228]).

--
Yusuke Endoh [e-mail]

=end

[mame (Yusuke Endoh)]

=begin
Hi,

2010/7/13 James Tucker [e-mail]:

On 13 Jul 2010, at 01:04, Roger Pack wrote:

If you 'gem install mongrel' on 1.9, it will fail (for a valid reason). The directory is not removed because that would prevent users from debugging the build process by reading 'gem_make.out' that is left there for this purpose.

Yeah I've run into this before. I even filed a bug for it with
rubygems, but nothing was done. Maybe Eric is too busy?

It is NOT a bug in rubygems. It's a bug in gem_prelude.rb. gem_prelude.rb is broken by design. I may have even marked that ticket as invalid for you.

gem_prelude.rb was imported by Eric himself at r14011.

commit 369697bcebcfd05a2b7d541d59a85fa529cfe6c9
Author: drbrain drbrain@b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Date: Sun Nov 25 03:26:36 2007 +0000

   Import fast-loading gem_prelude.rb from RubyGems.

   Import RubyGems r1516.

I wonder when and why he thought that gem_prelude.rb was different
product from rubygems itself, and that he was not a maintainer for
gem_prelude.rb.

--
Yusuke Endoh [e-mail]

=end

[evanphx (Evan Phoenix)]

=begin

On Jul 12, 2010, at 8:27 PM, Yusuke ENDOH wrote:

2010/7/13 Yehuda Katz [e-mail]:

If the problem is that ruby-core doesn't use RubyGems that much, and isn't willing to avoid breaking fundamental package manager properties, perhaps it's worth reconsidering including it in Ruby 1.9.

Agreed.

BTW, I don't see why you did not complain until rc1 is released.
I gave and noticed you a chance to test your product [ruby-core:30543].
In addition, the preview plan was announced in May [ruby-core:28665].

I'm sorry that I got involved in the situation so late. Yehuda and Aaron Patterson both approached me about the problem not long ago.

In fact, I heard from Shyouhei that you had noticed some trouble about
this issue in June (though I cannot know the detail because you did
not complain formally).

I'm unable to speak about how or when the bug was seen in the past, but regardless, it did go overlooked for too long. All we can do now is correct the situation.

You actually had a chance. In this case, I can't help but say that you
are too uncooperative, or even opposing to 1.9.2 release. This result
is what you asked for.

In this case I guess we could manage this issue, thanks to Evan's patch.
But, please please cooperate us in next time.

I asked Eric to official appoint me as the maintainer of gem_prelude so that future issues related to it can be handled swiftly. I'm happy to fulfill this duty.

My hope is that my patch goes into 1.9.2, and on trunk we can further refine the solution. In the coming weeks, I'll be asking ruby-core for input on what the core features of gem_prelude must be so that we can accomplish that goal without problems.

Yusuke, thank you for your patience in dealing with this bug. I know that it has cause d the 1.9.2 release process and you much grief, but I think we all now agree that fixing the bug in some way is necessary for 1.9.2.

Your humble gem_prelude maintainer,
Evan Phoenix

--
Yusuke Endoh [e-mail]

=end

[evanphx (Evan Phoenix)]

=begin

On Jul 12, 2010, at 8:38 PM, Yusuke ENDOH wrote:

Hi,

2010/7/13 Yukihiro Matsumoto [e-mail]:

I am sorry that I haven't followed this issue. As far as I
understand, this is caused by unfortunate timing. Eric was busy at
the critical timing

Not only at that timing. He is always busy.

[ruby-core:28689] [ruby-core:28781] [ruby-core:28926]
[ruby-core:30127] [ruby-core:30129] [ruby-core:30130]
[ruby-core:30151] [ruby-core:30189] [ruby-core:30456]
[ruby-core:30619] [ruby-core:30621] [ruby-core:30792]
[ruby-core:30907]

Of course I can't blame his busyness, but looking objectively,
he looks like ineligible for a maintainer.

Now that I have been appointed gem_prelude maintainer, my hope is that this will no longer be an issue.

I think 1.9.2 should run Rails3.

I hope too. I wanted them to know the very poor maintenance status
of rubygems and to do it themselves, as they are beneficiaries.

OTOH there are also many people who wait for 1.9.2 but don't need
rubygems nor rails.

We must not overlook users of rubygems and rails on 1.9.2. In the US as well as europe, almost all ruby users use rubygems and significant percentage also use rails.

I hope to use my presence to aid in rubygems maintenance status for 1.9.2 going forward.

If I understand what Yehuda told me
correctly, this issue could be a show stopper.

I don't agree to see the issue as a show stopper unless some active
developer is assigned to the issue. Otherwise it would take forever,
literally.

As an open source developer, I agree that an issue can only be a show stopper if there are active developers assigned to it. This is why I have become much more active in helping to resolve this situation.

In practice, I think Evan's patch is a meeting point, if Roger can
confirm the [BUG] does not occur on ruby_1_9_2 with Evan's patch.
If it does not occur on ruby_1_9_2, I also suspect it is due to lazy
sweep ([ruby-core:31228]).

I am happy to hear that we are getting close to figure out the source of the GC bug. Please let me know whatever I can do to help get my patch into 1.9.2.

Thank you.

• Evan

--
Yusuke Endoh [e-mail]

=end

[evanphx (Evan Phoenix)]

=begin

On Jul 12, 2010, at 8:47 PM, Yusuke ENDOH wrote:

Hi,

2010/7/13 James Tucker [e-mail]:

On 13 Jul 2010, at 01:04, Roger Pack wrote:

If you 'gem install mongrel' on 1.9, it will fail (for a valid reason). The directory is not removed because that would prevent users from debugging the build process by reading 'gem_make.out' that is left there for this purpose.

Yeah I've run into this before. I even filed a bug for it with
rubygems, but nothing was done. Maybe Eric is too busy?

It is NOT a bug in rubygems. It's a bug in gem_prelude.rb. gem_prelude.rb is broken by design. I may have even marked that ticket as invalid for you.

gem_prelude.rb was imported by Eric himself at r14011.

commit 369697bcebcfd05a2b7d541d59a85fa529cfe6c9
Author: drbrain drbrain@b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Date: Sun Nov 25 03:26:36 2007 +0000

 Import fast-loading gem_prelude.rb from RubyGems.

 Import RubyGems r1516.

I wonder when and why he thought that gem_prelude.rb was different
product from rubygems itself, and that he was not a maintainer for
gem_prelude.rb.

The situation with gem_prelude goes back many years now. Originally it was written by Rich Kilmer. I have discussed with Rich about gem_prelude at length to get an idea for what features it is meant to have.

Because 1.9 has been going on for a long time now, gem_prelude got stale. The bug we are all working hard to fix does not manifest right away, so I suspect that people may have either thought it was an issue with other parts of 1.9 and worked around it. Because gem_prelude is unique to 1.9, up to now it has not gotten the time and care we all now recognize it requires.

At some point, nobu began to work on gem_prelude. I do not know for sure, but perhaps Eric believed nobu was taking over as gem_prelude maintainer. This does not matter though. We know the bugs, we are now actively working to fix them.

I hope to guide gem_prelude more smoothly in the future.

• Evan

--
Yusuke Endoh [e-mail]

=end

[matz (Yukihiro Matsumoto)]

=begin
Hi,

In message "Re: [ruby-core:31236] Re: [Bug #3140] gem activation has changed between 1.8 and 1.9"
on Tue, 13 Jul 2010 12:38:32 +0900, Yusuke ENDOH [e-mail] writes:

|OTOH there are also many people who wait for 1.9.2 but don't need
|rubygems nor rails.

Do you mean THAT MANY enough to justify ignoring poor rubygems (thus
rails) users? I doubt.

|I don't agree to see the issue as a show stopper unless some active
|developer is assigned to the issue. Otherwise it would take forever,
|literally.

Now we have Evan. Thanks.

|In practice, I think Evan's patch is a meeting point, if Roger can
|confirm the [BUG] does not occur on ruby_1_9_2 with Evan's patch.
|If it does not occur on ruby_1_9_2, I also suspect it is due to lazy
|sweep ([ruby-core:31228]).

I think we have to separate the issue. If we don't see the problem on
1.9.2 we can consider it being fixed.

						matz.

=end

[mame (Yusuke Endoh)]

=begin
Hi,

2010/7/13 Yukihiro Matsumoto [e-mail]:

|I don't agree to see the issue as a show stopper unless some active
|developer is assigned to the issue. ?Otherwise it would take forever,
|literally.

Now we have Evan. Thanks.

Matz approved Evan as a maintainer.
I think Evan should have a commit bit. After that I'll update the
maintainer list.

|In practice, I think Evan's patch is a meeting point, if Roger can
|confirm the [BUG] does not occur on ruby_1_9_2 with Evan's patch.
|If it does not occur on ruby_1_9_2, I also suspect it is due to lazy
|sweep ([ruby-core:31228]).

I think we have to separate the issue. If we don't see the problem on
1.9.2 we can consider it being fixed.

Almost all things committed to ruby_1_9_2 are backports from trunk.
So I believe there is no bug that is fixed in ruby_1_9_2 but not in
trunk.
If the problem does not occur on 1.9.2, I bet that some bugs are
committed to trunk (or that the problem is poorly-reproducible).

--
Yusuke Endoh [e-mail]

=end

[raggi (James Tucker)]

=begin

On 13 Jul 2010, at 16:16, Roger Pack wrote:

I'll try with 1_9_2 next.

Bug doesn't appear to surface with 1.9.2 branch.
Phew!

@James: http://rubyforge.org/tracker/index.php?func=detail&aid=26792&group_id=126&atid=575

gem list doesn't use prelude, require does. Eric didn't rep. the problem there. I will correct the ticket there, but I should pass this on to Evan in reality.

=end

[mame (Yusuke Endoh)]

=begin
2010/7/14 Roger Pack [e-mail]:

I'll try with 1_9_2 next.

Bug doesn't appear to surface with 1.9.2 branch.
Phew!

Good. I've committed Evan's patch to ruby_1_9_2 (r28634).
If you notice any problem, please let us know.

Thank you all!

--
Yusuke Endoh [e-mail]

=end

[evanphx (Evan Phoenix)]

=begin

On Jul 13, 2010, at 8:57 AM, Yusuke ENDOH wrote:

2010/7/14 Roger Pack [e-mail]:

I'll try with 1_9_2 next.

Bug doesn't appear to surface with 1.9.2 branch.
Phew!

Good. I've committed Evan's patch to ruby_1_9_2 (r28634).
If you notice any problem, please let us know.

Wonderful! Thank you Yusuke! I'll keep a close eye on things to watch for anyone reporting bugs.

Did the lazy sweep bug get fixed?

• Evan

Thank you all!

--
Yusuke Endoh [e-mail]

=end

[mame (Yusuke Endoh)]

=begin
2010/7/14 Evan Phoenix [e-mail]:

Did the lazy sweep bug get fixed?

Not yet, I guess. But ruby_1_9_2 is not affected because the
lazy sweep feature is committed only to trunk. It is not a
concern of 1.9.2 release.

FYI, Narihiro, an author of lazy sweep, is investigating now
[ruby-core:31239].

--
Yusuke Endoh [e-mail]

=end

[matz (Yukihiro Matsumoto)]

=begin
Hi,

In message "Re: [ruby-core:31258] Re: [Bug #3140] gem activation has changed between 1.8 and 1.9"
on Wed, 14 Jul 2010 00:31:40 +0900, Yusuke ENDOH [e-mail] writes:

|> Now we have Evan. Thanks.
|
|Matz approved Evan as a maintainer.
|I think Evan should have a commit bit. After that I'll update the
|maintainer list.

Agreed. Evan, could you send your PGP signed ssh2 public key to
[e-mail]? I hesitate to ask you again, but it's
needed.

						matz.

=end

[evanphx (Evan Phoenix)]

=begin

On Jul 13, 2010, at 9:56 AM, Yukihiro Matsumoto wrote:

Hi,

In message "Re: [ruby-core:31258] Re: [Bug #3140] gem activation has changed between 1.8 and 1.9"
on Wed, 14 Jul 2010 00:31:40 +0900, Yusuke ENDOH [e-mail] writes:

|> Now we have Evan. Thanks.
|
|Matz approved Evan as a maintainer.
|I think Evan should have a commit bit. After that I'll update the
|maintainer list.

Agreed. Evan, could you send your PGP signed ssh2 public key to
[e-mail]? I hesitate to ask you again, but it's
needed.

Sorry for not sending in my key before. I got distracted setting up PGP and forgot. I have just emailed the details to [e-mail].

 					matz.

=end

[evanphx (Evan Phoenix)]

=begin

On Jul 16, 2010, at 9:07 AM, Roger Pack wrote:

Wonderful! Thank you Yusuke! I'll keep a close eye on things to watch for anyone reporting bugs.

Evan, could you also merge in Nobu's patch to load full rubygems when
users do an

gem 'xxx'

otherwise gem_prelude is still quite unsafe (as pointed out by James,
I only understood it just now).

Also could that change (after committed) be backported to 1.9.2 branch, please?
I just want to clear up confusion for end users, and it causes some
obscure and hard to understand bugs.
That would be awesome!

No, I can't backport anything to 1.9.2. It's unlikely this will happen for 1.9.2 honestly. I know that my patch for 1.9.2 is less than ideal, but it at least works around the problem in some ways.

If Yusuke indicates it would be fine to revise it, we can do that. But I think that they don't want to change 1.9.2 any more.

-r

=end