Project

General

Profile

Actions

Bug #15461

closed

ruby-2.6.0.tar.gz seems incorrectly packed

Added by valich (Valentin Fondaratov) over 2 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
[ruby-core:90701]

Description

  1. Download ftp.ruby-lang.org/pub/ruby/2.6/ruby-2.6.0.tar.gz

  2. Run from the console:

ruby -e 'require "zlib";require "minitar";Minitar.unpack(Zlib::GzipReader.new(File.open("ruby-2.6.0.tar.gz", "rb")), "x")'

It fails with an exception:

Traceback (most recent call last):
    17: from -e:1:in `<main>'
    16: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar.rb:249:in `unpack'
    15: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar/input.rb:27:in `open'
    14: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar.rb:256:in `block in unpack'
    13: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar/input.rb:90:in `each_entry'
    12: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar/reader.rb:207:in `each_entry'
    11: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar/reader.rb:207:in `loop'
    10: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar/reader.rb:226:in `block in each_entry'
     9: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar/input.rb:91:in `block in each_entry'
     8: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar.rb:258:in `block (2 levels) in unpack'
     7: from /home/user/.rvm/gems/ruby-2.5.0/gems/minitar-0.7/lib/archive/tar/minitar/input.rb:162:in `extract_entry'
     6: from /home/user/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/fileutils.rb:193:in `mkdir_p'
     5: from /home/user/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/fileutils.rb:193:in `each'
     4: from /home/user/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/fileutils.rb:208:in `block in mkdir_p'
     3: from /home/user/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/fileutils.rb:208:in `reverse_each'
     2: from /home/user/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/fileutils.rb:210:in `block (2 levels) in mkdir_p'
     1: from /home/user/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/fileutils.rb:229:in `fu_mkdir'
/home/user/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/fileutils.rb:229:in `mkdir': File exists @ dir_s_mkdir - x/ruby-2.6.0/libexec (Errno::EEXIST)

It seems flags are set incorrectly in the package. Same command for previous rubies tgzs works well.

Updated by shevegen (Robert A. Heiler) over 2 years ago

This confuses me a bit though; using "tar" from
ftp://ftp.gnu.org/gnu/tar/tar-1.30.tar.xz repackaging works for me. Is this not rather
a behaviour problem of minitar if it chokes whereas tar works fine? I just repackaged
the .tar.gz into .tar.xz locally here via tar and that works.

Updated by knu (Akinori MUSHA) over 2 years ago

Maybe this issue should be escalated to the minitar project.

One difference I found between the 2.5.3 and 2.6.0 tarballs is that the latter starts with the entry ruby-2.6.0/libexec/ whereas the former starts with ruby-2.5.3/, and minitar stucks at trying mkdir libexec when somehow there's already an empty regular file named libexec there.

Updated by naruse (Yui NARUSE) over 2 years ago

  • Status changed from Open to Third Party's Issue

It's considered a minitar's issue.

Updated by valich (Valentin Fondaratov) over 2 years ago

OK, I've made the same experiment from the command line:

user@UNIT-867:~/Downloads$ gunzip ruby-2.6.0.tar.gz 
user@UNIT-867:~/Downloads$ tar -xf ruby-2.6.0.tar ruby-2.6.0
tar: ruby-2.6.0: Not found in archive
tar: Exiting with failure status due to previous errors
user@UNIT-867:~/Downloads$ gunzip ruby-2.5.3.tar.gz 
user@UNIT-867:~/Downloads$ tar -xf ruby-2.5.3.tar ruby-2.5.3

command-line tar fails, too, so I am not sure this is minitar issue.

Interestingly, normal tar -xzf ruby-2.6.0.tar.gz works.

Updated by mame (Yusuke Endoh) over 2 years ago

What OS are you using? What version of tar are you using?

Updated by valich (Valentin Fondaratov) over 2 years ago

mame (Yusuke Endoh) wrote:

What OS are you using? What version of tar are you using?

tar (GNU tar) 1.29,
Linux UNIT-867 4.15.0-43-generic #46-Ubuntu SMP Thu Dec 6 14:45:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux aka Ubuntu 18.04.1

Updated by mame (Yusuke Endoh) over 2 years ago

I tried docker run --rm -ti ubuntu:18.04 and it works. Why...

root@07f1505cc1d7:/# uname -a
Linux 07f1505cc1d7 4.18.0-13-generic #14-Ubuntu SMP Wed Dec 5 09:04:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
root@07f1505cc1d7:/# cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"
root@07f1505cc1d7:/# tar --version
tar (GNU tar) 1.29
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by John Gilmore and Jay Fenlason.
root@07f1505cc1d7:/# wget https://ftp.ruby-lang.org/pub/ruby/2.6/ruby-2.6.0.tar.gz
--2018-12-25 13:02:15--  https://ftp.ruby-lang.org/pub/ruby/2.6/ruby-2.6.0.tar.gz
Resolving ftp.ruby-lang.org (ftp.ruby-lang.org)... 151.101.129.178, 151.101.193.178, 151.101.1.178, ...
Connecting to ftp.ruby-lang.org (ftp.ruby-lang.org)|151.101.129.178|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 16687800 (16M) [application/x-tar]
Saving to: 'ruby-2.6.0.tar.gz'

ruby-2.6.0.tar.gz         100%[=====================================>]  15.91M  3.09MB/s    in 7.4s    

2018-12-25 13:02:23 (2.15 MB/s) - 'ruby-2.6.0.tar.gz' saved [16687800/16687800]

root@07f1505cc1d7:/# rm -rf ruby-2.6.0 ruby-2.6.0.tar
root@07f1505cc1d7:/# gunzip ruby-2.6.0.tar.gz 
root@07f1505cc1d7:/# tar -xf ruby-2.6.0.tar ruby-2.6.0

Updated by knu (Akinori MUSHA) over 2 years ago

After investigation, I found the 2.6.0 tar file has the "typeflag" of '0' for each directory entry instead of '5' which modern tar implementations should use.
The reason major implementations like GNU tar and libarchive-based tar can extract the directories with typeflag '0' is because they have special treatment (for interoperability and compatibility) to treat a file with a trailing slash as directory regardless of the typeflag, which minitar is seemingly missing in its implementation.

libarchive: https://github.com/libarchive/libarchive/blob/master/libarchive/archive_read_support_format_tar.c#L566
GNU tar: http://git.savannah.gnu.org/cgit/tar.git/tree/src/extract.c#n1543

So, my conclusion is that this can be treated as a packaging issue while major tar implementations can deal with it.

Actions #9

Updated by naruse (Yui NARUSE) over 2 years ago

  • Status changed from Third Party's Issue to Open

Updated by mame (Yusuke Endoh) over 2 years ago

knu (Akinori MUSHA) wrote:

After investigation, I found the 2.6.0 tar file has the "typeflag" of '0' for each directory entry instead of '5' which modern tar implementations should use.

This issue has been fixed at r66553. 2.6.1 will be a sane tarball.

Updated by valich (Valentin Fondaratov) over 2 years ago

Thank you for swift fixing!

Actions #12

Updated by naruse (Yui NARUSE) over 2 years ago

  • Status changed from Open to Closed

Applied in changeset commit:ruby-git|477fe4ac23592fe0a835b3b8c15594315bf39d22.


merge revision(s) 66553:

    Set typeflag [Bug #15461]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_6@66562 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Updated by knu (Akinori MUSHA) over 2 years ago

I confirmed the above PR fixes the problem as below:

% gem specific_install https://github.com/knu/minitar dir_with_typeflag_0
...
http installing from https://github.com/knu/minitar
...
Successfully installed
% ruby -e 'require "zlib";require "minitar";Minitar.unpack(Zlib::GzipReader.new(File.open("ruby-2.6.0.tar.gz", "rb")), "x")'
% ls -al x/ruby-2.6.0
total 12016
drwxr-xr-x  208 knu  staff     6656 Dec 26 13:41 .
drwxr-xr-x    3 knu  staff       96 Dec 26 13:41 ..
-rw-r--r--    1 knu  staff      506 Dec 26 13:41 .document
-rw-r--r--    1 knu  staff      336 Dec 26 13:41 .editorconfig
...

Updated by austin (Austin Ziegler) over 2 years ago

I have released an update to minitar that should fix this. Thanks, knu! https://rubygems.org/gems/minitar/versions/0.8

Actions

Also available in: Atom PDF