Bug #888: zlib 1.2.3 does not work with Rubygems 1.3.1 (in Ruby 1.9.1) on Windows

• Status: Closed
• Assignee: usa (Usaku NAKAMURA)
• Target version: 1.9.1

[Chauk-Mean (Chauk-Mean Proum)]

=begin
Hi,

I built successfully zlib 1.1.4 (zlib-1.1.4-1-src.zip from http://jarp.does.notwork.org/win32) and ruby-1.9.1-preview2 from scratch with Visual C++ 2008 Express Edition SP1 on Windows XP SP2.
Ruby and Rubygems work well. I have been able to install locally (-l flag) rspec-1.1.11.gem for example.

I tried to create a new build with zlib 1.2.3 (zlib123.zip from http://zlib.net).
Now, the installation of a gem raises the following error :

D:\HOME>gem19 install -l rspec
C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/package/tar_input.rb:212: [BUG] Segmenta
tion fault
ruby 1.9.1 (2008-12-01 revision 20438) [i386-mswin32_90]

-- control frame ----------
c:0034 p:---- s:0149 b:0149 l:000148 d:000148 CFUNC :inflate
c:0033 p:0138 s:0145 b:0144 l:000143 d:000143 METHOD C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/package/tar_input.rb:212
c:0032 p:0034 s:0137 b:0137 l:000118 d:000136 BLOCK C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/package/tar_input.rb:123
c:0031 p:0114 s:0133 b:0133 l:000121 d:000132 BLOCK C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/package/tar_reader.rb:46
c:0030 p:---- s:0127 b:0127 l:000126 d:000126 FINISH
c:0029 p:---- s:0125 b:0125 l:000124 d:000124 CFUNC :loop
c:0028 p:0011 s:0122 b:0122 l:000121 d:000121 METHOD C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/package/tar_reader.rb:37
c:0027 p:0012 s:0119 b:0119 l:000118 d:000118 METHOD C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/package/tar_input.rb:121
c:0026 p:0046 s:0115 b:0115 l:0025b4 d:000110 BLOCK C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/format.rb:71
c:0025 p:0027 s:0112 b:0112 l:000111 d:000111 METHOD C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/package/tar_input.rb:20
c:0024 p:0095 s:0105 b:0105 l:000104 d:000104 METHOD C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/package.rb:56
c:0023 p:0043 s:0097 b:0097 l:0025b4 d:0025b4 METHOD C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/format.rb:67
c:0022 p:0018 s:0090 b:0090 l:000081 d:000089 BLOCK C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/format.rb:51
c:0021 p:---- s:0089 b:0089 l:000088 d:000088 FINISH
c:0020 p:---- s:0087 b:0087 l:000086 d:000086 CFUNC :open
c:0019 p:0157 s:0082 b:0082 l:000081 d:000081 METHOD C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/format.rb:50
c:0018 p:0209 s:0076 b:0076 l:000075 d:000075 METHOD C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/installer.rb:114
c:0017 p:---- s:0069 b:0069 l:000068 d:000068 FINISH
c:0016 p:---- s:0067 b:0067 l:000066 d:000066 CFUNC :new
c:0015 p:0214 s:0062 b:0062 l:001e9c d:00263c BLOCK C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/dependency_installer.rb:236
c:0014 p:---- s:0056 b:0056 l:000055 d:000055 FINISH
c:0013 p:---- s:0054 b:0054 l:000053 d:000053 CFUNC :each
c:0012 p:0101 s:0051 b:0051 l:001e9c d:001e9c METHOD C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/dependency_installer.rb:219
c:0011 p:0049 s:0046 b:0046 l:000037 d:000045 BLOCK C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/commands/install_command.rb:86
c:0010 p:---- s:0043 b:0043 l:000042 d:000042 FINISH
c:0009 p:---- s:0041 b:0041 l:000040 d:000040 CFUNC :each
c:0008 p:0241 s:0038 b:0038 l:000037 d:000037 METHOD C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/commands/install_command.rb:83
c:0007 p:0071 s:0031 b:0031 l:000030 d:000030 METHOD C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/command.rb:136
c:0006 p:0194 s:0027 b:0027 l:000026 d:000026 METHOD C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/command_manager.rb:105
c:0005 p:0013 s:0021 b:0021 l:000020 d:000020 METHOD C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/command_manager.rb:75
c:0004 p:0089 s:0016 b:0016 l:000015 d:000015 METHOD C:/opt/ruby19/lib/ruby19/1.
9.1/rubygems/gem_runner.rb:39
c:0003 p:0207 s:0009 b:0009 l:000008 d:000008 TOP C:/opt/ruby19/bin/gem19.bat
:32
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:0000 s:0002 b:0002 l:000001 d:000001 TOP :300

DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/package/tar_input.rb:212:in inf late'" DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/package/tar_input.rb:212:in zip
ped_stream'"
DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/package/tar_input.rb:123:in blo ck in each'" DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/package/tar_reader.rb:46:in blo
ck in each'"
DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/package/tar_reader.rb:37:in loo p'" DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/package/tar_reader.rb:37:in eac
h'"
DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/package/tar_input.rb:121:in eac h'" DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/format.rb:71:in block in from_i
o'"
DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/package/tar_input.rb:20:in open '" DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/package.rb:56:in open'"
DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/format.rb:67:in from_io'" DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/format.rb:51:in block in from_f
ile_by_path'"
DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/format.rb:50:in open'" DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/format.rb:50:in from_file_by_pa
th'"
DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/installer.rb:114:in initialize' " DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/dependency_installer.rb:236:in
new'"
DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/dependency_installer.rb:236:in block in install'" DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/dependency_installer.rb:219:in
each'"
DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/dependency_installer.rb:219:in install'" DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/commands/install_command.rb:86:i nblock in execute'"
DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/commands/install_command.rb:83:i
n each'" DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/commands/install_command.rb:83:i n execute'"
DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/command.rb:136:in invoke'" DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/command_manager.rb:105:in proce
ss_args'"
DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/command_manager.rb:75:in run'" DBG> : "C:/opt/ruby19/lib/ruby19/1.9.1/rubygems/gem_runner.rb:39:in run'"
DBG> : "C:/opt/ruby19/bin/gem19.bat:32:in `'"

Issues have already been reported with zlib-1.2.3 and rubygems-1.2.x on Windows but I thought these have been fixed.

The detailed steps for the build :

zlib-1.2.3 build :
nmake -f win32\Makefile.msc
nmake -f win32\Makefile.msc test
The tests are OK.

zlib-1.2.3 installation :

• copy of zlib.h and zconf.h in a folder seen in INCLUDE
• copy of zlib.lib, zdll.lib and zdll.exp in a folder seen in LIB
• copy of zlib1.dll in a folder seen in PATH

ruby-1.9.1-preview2 build :
win32\configure --prefix=C:/opt/ruby19 --program-suffix=19
nmake
nmake install

Regards.

Chauk-Mean.
=end

[Chauk-Mean (Chauk-Mean Proum)]

=begin
Oups !
The title is a bit erroneous. It should have been :
zlib 1.2.3 is not supported by Rubygems 1.3.1 (in Ruby 1.9.1) on Windows.
=end

[yugui (Yuki Sonoda)]

• Assignee set to usa (Usaku NAKAMURA)
• Priority changed from Normal to 5

=begin

=end

[yugui (Yuki Sonoda)]

• Due date set to 12/24/2008

=begin

=end

[yugui (Yuki Sonoda)]

• Status changed from Open to Rejected

=begin
can't reproduce
=end

[Chauk-Mean (Chauk-Mean Proum)]

=begin
Sorry to bother with this issue ... but this really does not work with Visual C++ 2008.
I completely rebuilt zlib and ruby as described on another machine and I had the same problem.

Conversely, a rebuild with Visual C++ 6.0 works ! I have been able to install a gem without any problem.

Have you tried to reproduce with Visual C++ 2008 ?
Can you give the steps you followed if they are different from mine ?

Thanks.

=end

[yugui (Yuki Sonoda)]

• Status changed from Rejected to Open

=begin

=end

=begin
I can reproduce this issue, and I think following method might
have fixed that.

1. open ``c:/opt/ruby19/lib/ruby19/1.9.1/rubygems/package/tar_input.rb''

2. edit ``zipped_stream(entry)'' (line 203--216) as follows:

   def zipped_stream(entry)
   if defined? Rubinius then
   zis = Zlib::GzipReader.new entry
   dis = zis.read
   is = StringIO.new(dis)
   else
   # This is Jamis Buck's Zlib workaround for some unknown issue
   entry.read(10) # skip the gzip header
   zis = Zlib::Inflate.new(-Zlib::MAX_WBITS)

   ######### EDIT START ###
   #is = StringIO.new(zis.inflate(entry.read))
   
   __buf = ""
   while __temp = entry.read( 128 * 1024 ) do
     __buf << zis.inflate( __temp )
   end
   
   is = StringIO.new( __buf )
   ######### EDIT END #####
   

   end
   ensure
   zis.finish if zis
   end

On my machine this method seems like working, but I'm not sure it works correctly.
Could you confirm this?

Regards.

Y. MISE

=end

[Chauk-Mean (Chauk-Mean Proum)]

=begin
This works perfectly !
Thanks a lot.

Regards.

Chauk-Mean.
=end

[luislavena (Luis Lavena)]

=begin
If you're building Ruby with VC9 (VS2008) then you need to also build zlib with VC9.

The problems you're experiencing can (and most of the cases are) related to cross CRT issues.

zlib is linked to MSVCRT while your Ruby links to MSVCR90. Freeing and allocating memory across CRTs generates errors, segfaults and other weird results.

Daniel Berger had problems building zlib with VC8/9 too. Why don't you try MinGW (GCC)?

=end

[Chauk-Mean (Chauk-Mean Proum)]

=begin

If you're building Ruby with VC9 (VS2008) then you need to also build zlib with VC9.
Of course, that's what I've done. Please take a look at my initial bug report.

Daniel Berger had problems building zlib with VC8/9 too.
I had no problem with building zlib in itself.
Ruby works also fine with zlib (all tests in test/zlib passed successfully).
The problem I had was related with Rubygems.
This is now fixed thanks to the patch from Yasuhiro.

Why don't you try MinGW (GCC)?
I also use MinGW and I have other problems at the moment related with wxWidgets ...
I patched MinGW and wxWidgets for the GDI+ support, some samples work fine, others don't work (e.g. RichTextCtrl).
Hopefully, this doesn't prevent the correct build of the wxRuby gem.

Conversely, I have no problem with the VS 2008 build of wxWidgets.

Chauk-Mean.

=end

=begin
well.. I think this might be caused by shortage of
buffer or something.

If I change the line:
while __temp = entry.read( 128 * 1024 ) do
to:
while __temp = entry.read( 512 * 1024 ) do

then, the latter causes segmentation fault on my machine.

Size of rspec-1.1.11.gem is approx. 260KB, so, perhaps
buffer of zis.inflate() is not enough to handle 260KB??
=end

[yugui (Yuki Sonoda)]

• Target version changed from 1.9.1 Release Candidate to 1.9.1 RC2

=begin

=end

[drbrain (Eric Hodel)]

=begin
On Dec 29, 2008, at 07:14 AM, Yasuhiro MISE wrote:

2. edit ``zipped_stream(entry)'' (line 203--216) as follows:

def zipped_stream(entry)
if defined? Rubinius then
zis = Zlib::GzipReader.new entry
dis = zis.read
is = StringIO.new(dis)
else
# This is Jamis Buck's Zlib workaround for some unknown issue
entry.read(10) # skip the gzip header
zis = Zlib::Inflate.new(-Zlib::MAX_WBITS)

######### EDIT START ###
#is = StringIO.new(zis.inflate(entry.read))

__buf = ""
while __temp = entry.read( 128 * 1024 ) do
  __buf << zis.inflate( __temp )
end

is = StringIO.new( __buf )
######### EDIT END #####

end
ensure
zis.finish if zis
end

For what it's worth, I'd really like to see the root cause of this bug
fixed so this method can be returned to a simpler form:

def zipped_stream(entry)
zis = Zlib::GzipReader.new entry
dis = zis.read
is = StringIO.new(dis)
ensure
zis.finish if zis
end

Or even:

def zipped_stream(entry)
Zlib::GzipReader.new entry
end

I'm not sure where the bug actually exists, but I'd really like to see
the bug fixed. I'm fairly certain the bug doesn't exist in RubyGems
and I'm tired of having to continually apply bandaids to RubyGems.

While we're on the subject, what help can I provide so I can
eventually get back to the bandaid-free version?

=end

[luislavena (Luis Lavena)]

=begin
On Tue, Dec 30, 2008 at 8:15 PM, Eric Hodel [e-mail] wrote:

On Dec 29, 2008, at 07:14 AM, Yasuhiro MISE wrote:

2. edit ``zipped_stream(entry)'' (line 203--216) as follows:

def zipped_stream(entry)
if defined? Rubinius then
zis = Zlib::GzipReader.new entry
dis = zis.read
is = StringIO.new(dis)
else

This is Jamis Buck's Zlib workaround for some unknown issue

entry.read(10) # skip the gzip header
zis = Zlib::Inflate.new(-Zlib::MAX_WBITS)

######### EDIT START ###
#is = StringIO.new(zis.inflate(entry.read))

__buf = ""
while __temp = entry.read( 128 * 1024 ) do
__buf << zis.inflate( __temp )
end

is = StringIO.new( __buf )
######### EDIT END #####

end
ensure
zis.finish if zis
end

For what it's worth, I'd really like to see the root cause of this bug fixed
so this method can be returned to a simpler form:

def zipped_stream(entry)
zis = Zlib::GzipReader.new entry
dis = zis.read
is = StringIO.new(dis)
ensure
zis.finish if zis
end

Or even:

def zipped_stream(entry)
Zlib::GzipReader.new entry
end

I'm not sure where the bug actually exists, but I'd really like to see the
bug fixed. I'm fairly certain the bug doesn't exist in RubyGems and I'm
tired of having to continually apply bandaids to RubyGems.

While we're on the subject, what help can I provide so I can eventually get
back to the bandaid-free version?

Gregory Brown offered backport the zlib tests from 1.9 to 1.8 in
[ruby-core:20602]

Maybe those can help pinpoint the root of the problem.

I offer my Windows resources to see if the problem is located on this
side of the bench.

--
Luis Lavena
AREA 17

Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exupéry

=end

=begin
Eric Hodel wrote:

For what it's worth, I'd really like to see the root cause of this bug
fixed so this method can be returned to a simpler form:

def zipped_stream(entry)
zis = Zlib::GzipReader.new entry
dis = zis.read
is = StringIO.new(dis)
ensure
zis.finish if zis
end

Or even:

def zipped_stream(entry)
Zlib::GzipReader.new entry
end

Thanks your advice, and yes, each forms works fine!
Hope that Chauk-Mean can also confirm that these forms fix this issue.

I'm fairly certain the bug doesn't exist in RubyGems

I agree, and I also agree to Luis's indication:

Gregory Brown offered backport the zlib tests from 1.9 to 1.8 in
[ruby-core:20602]

Maybe those can help pinpoint the root of the problem.

Y. MISE
=end

[Chauk-Mean (Chauk-Mean Proum)]

=begin
@luis (Luis Lopez)

Gregory Brown offered backport the zlib tests from 1.9 to 1.8 in [ruby-core:20602]
Maybe those can help pinpoint the root of the problem.

I'm running ruby-1.9.1 from the beginning and as I indicated in a previous message, all zlib tests pass successfully.

@eric (Eric Anderson)

I'm not sure where the bug actually exists, but I'd really like to see
the bug fixed. I'm fairly certain the bug doesn't exist in RubyGems
and I'm tired of having to continually apply bandaids to RubyGems.

I'm sorry to "have accused" Rubygems for this bug. But the zlib tests are OK.
May be there is a missing test case in zlib ?

@eric (Eric Anderson) and @yasuhiro (Yasuhiro Yoshida)

For what it's worth, I'd really like to see the root cause of this bug
fixed so this method can be returned to a simpler form:

def zipped_stream(entry)
zis = Zlib::GzipReader.new entry
dis = zis.read
is = StringIO.new(dis)
ensure
zis.finish if zis
end

Or even:

def zipped_stream(entry)
Zlib::GzipReader.new entry
end

Yes, the two forms work also fine for me.
The second looks really better (simple is better :-)).

Chauk-Mean.

=end

[yugui (Yuki Sonoda)]

• Target version changed from 1.9.1 RC2 to 1.9.1

=begin

=end

[usa (Usaku NAKAMURA)]

• Status changed from Open to Closed
• % Done changed from 0 to 100

=begin
Applied in changeset r21859.
=end