Bug #15461
closedruby-2.6.0.tar.gz seems incorrectly packed
Description
-
Download
ftp.ruby-lang.org/pub/ruby/2.6/ruby-2.6.0.tar.gz
-
Run from the console:
ruby -e 'require "zlib";require "minitar";Minitar.unpack(Zlib::GzipReader.new(File.open("ruby-2.6.0.tar.gz", "rb")), "x")'
It fails with an exception:
Traceback (most recent call last):
17: from -e:1:in `<main>'
16: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar.rb:249:in `unpack'
15: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar/input.rb:27:in `open'
14: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar.rb:256:in `block in unpack'
13: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar/input.rb:90:in `each_entry'
12: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar/reader.rb:207:in `each_entry'
11: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar/reader.rb:207:in `loop'
10: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar/reader.rb:226:in `block in each_entry'
9: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar/input.rb:91:in `block in each_entry'
8: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar.rb:258:in `block (2 levels) in unpack'
7: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar/input.rb:162:in `extract_entry'
6: from /home/user/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/fileutils.rb:193:in `mkdir_p'
5: from /home/user/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/fileutils.rb:193:in `each'
4: from /home/user/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/fileutils.rb:208:in `block in mkdir_p'
3: from /home/user/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/fileutils.rb:208:in `reverse_each'
2: from /home/user/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/fileutils.rb:210:in `block (2 levels) in mkdir_p'
1: from /home/user/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/fileutils.rb:229:in `fu_mkdir'
/home/user/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/fileutils.rb:229:in `mkdir': File exists @ dir_s_mkdir - x/ruby-2.6.0/libexec (Errno::EEXIST)
It seems flags are set incorrectly in the package. Same command for previous rubies tgzs works well.
Updated by shevegen (Robert A. Heiler) almost 6 years ago
This confuses me a bit though; using "tar" from
ftp://ftp.gnu.org/gnu/tar/tar-1.30.tar.xz repackaging works for me. Is this not rather
a behaviour problem of minitar if it chokes whereas tar works fine? I just repackaged
the .tar.gz into .tar.xz locally here via tar and that works.
Updated by knu (Akinori MUSHA) almost 6 years ago
Maybe this issue should be escalated to the minitar project.
One difference I found between the 2.5.3 and 2.6.0 tarballs is that the latter starts with the entry ruby-2.6.0/libexec/
whereas the former starts with ruby-2.5.3/
, and minitar stucks at trying mkdir libexec
when somehow there's already an empty regular file named libexec
there.
Updated by naruse (Yui NARUSE) almost 6 years ago
- Status changed from Open to Third Party's Issue
It's considered a minitar's issue.
Updated by valich (Valentin Fondaratov) almost 6 years ago
OK, I've made the same experiment from the command line:
user@UNIT-867:~/Downloads$ gunzip ruby-2.6.0.tar.gz
user@UNIT-867:~/Downloads$ tar -xf ruby-2.6.0.tar ruby-2.6.0
tar: ruby-2.6.0: Not found in archive
tar: Exiting with failure status due to previous errors
user@UNIT-867:~/Downloads$ gunzip ruby-2.5.3.tar.gz
user@UNIT-867:~/Downloads$ tar -xf ruby-2.5.3.tar ruby-2.5.3
command-line tar fails, too, so I am not sure this is minitar
issue.
Interestingly, normal tar -xzf ruby-2.6.0.tar.gz
works.
Updated by mame (Yusuke Endoh) almost 6 years ago
What OS are you using? What version of tar are you using?
Updated by valich (Valentin Fondaratov) almost 6 years ago
mame (Yusuke Endoh) wrote:
What OS are you using? What version of tar are you using?
tar (GNU tar) 1.29
,
Linux UNIT-867 4.15.0-43-generic #46-Ubuntu SMP Thu Dec 6 14:45:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
aka Ubuntu 18.04.1
Updated by mame (Yusuke Endoh) almost 6 years ago
I tried docker run --rm -ti ubuntu:18.04
and it works. Why...
root@07f1505cc1d7:/# uname -a
Linux 07f1505cc1d7 4.18.0-13-generic #14-Ubuntu SMP Wed Dec 5 09:04:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
root@07f1505cc1d7:/# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"
root@07f1505cc1d7:/# tar --version
tar (GNU tar) 1.29
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by John Gilmore and Jay Fenlason.
root@07f1505cc1d7:/# wget https://ftp.ruby-lang.org/pub/ruby/2.6/ruby-2.6.0.tar.gz
--2018-12-25 13:02:15-- https://ftp.ruby-lang.org/pub/ruby/2.6/ruby-2.6.0.tar.gz
Resolving ftp.ruby-lang.org (ftp.ruby-lang.org)... 151.101.129.178, 151.101.193.178, 151.101.1.178, ...
Connecting to ftp.ruby-lang.org (ftp.ruby-lang.org)|151.101.129.178|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 16687800 (16M) [application/x-tar]
Saving to: 'ruby-2.6.0.tar.gz'
ruby-2.6.0.tar.gz 100%[=====================================>] 15.91M 3.09MB/s in 7.4s
2018-12-25 13:02:23 (2.15 MB/s) - 'ruby-2.6.0.tar.gz' saved [16687800/16687800]
root@07f1505cc1d7:/# rm -rf ruby-2.6.0 ruby-2.6.0.tar
root@07f1505cc1d7:/# gunzip ruby-2.6.0.tar.gz
root@07f1505cc1d7:/# tar -xf ruby-2.6.0.tar ruby-2.6.0
Updated by knu (Akinori MUSHA) almost 6 years ago
After investigation, I found the 2.6.0 tar file has the "typeflag" of '0' for each directory entry instead of '5' which modern tar implementations should use.
The reason major implementations like GNU tar and libarchive-based tar can extract the directories with typeflag '0' is because they have special treatment (for interoperability and compatibility) to treat a file with a trailing slash as directory regardless of the typeflag, which minitar is seemingly missing in its implementation.
libarchive: https://github.com/libarchive/libarchive/blob/master/libarchive/archive_read_support_format_tar.c#L566
GNU tar: http://git.savannah.gnu.org/cgit/tar.git/tree/src/extract.c#n1543
So, my conclusion is that this can be treated as a packaging issue while major tar implementations can deal with it.
Updated by naruse (Yui NARUSE) almost 6 years ago
- Status changed from Third Party's Issue to Open
Updated by mame (Yusuke Endoh) almost 6 years ago
knu (Akinori MUSHA) wrote:
After investigation, I found the 2.6.0 tar file has the "typeflag" of '0' for each directory entry instead of '5' which modern tar implementations should use.
This issue has been fixed at r66553. 2.6.1 will be a sane tarball.
Updated by valich (Valentin Fondaratov) almost 6 years ago
Thank you for swift fixing!
Updated by naruse (Yui NARUSE) almost 6 years ago
- Status changed from Open to Closed
Applied in changeset commit:ruby-git|477fe4ac23592fe0a835b3b8c15594315bf39d22.
merge revision(s) 66553:
Set typeflag [Bug #15461]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_6@66562 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Updated by knu (Akinori MUSHA) almost 6 years ago
Just submitted a PR to the third party: https://github.com/halostatue/minitar/pull/36
Updated by knu (Akinori MUSHA) almost 6 years ago
I confirmed the above PR fixes the problem as below:
% gem specific_install https://github.com/knu/minitar dir_with_typeflag_0
...
http installing from https://github.com/knu/minitar
...
Successfully installed
% ruby -e 'require "zlib";require "minitar";Minitar.unpack(Zlib::GzipReader.new(File.open("ruby-2.6.0.tar.gz", "rb")), "x")'
% ls -al x/ruby-2.6.0
total 12016
drwxr-xr-x 208 knu staff 6656 Dec 26 13:41 .
drwxr-xr-x 3 knu staff 96 Dec 26 13:41 ..
-rw-r--r-- 1 knu staff 506 Dec 26 13:41 .document
-rw-r--r-- 1 knu staff 336 Dec 26 13:41 .editorconfig
...
Updated by austin (Austin Ziegler) almost 6 years ago
I have released an update to minitar that should fix this. Thanks, knu! https://rubygems.org/gems/minitar/versions/0.8