Bug #15304: Package build with extensions is not reproducible

• Status: Third Party's Issue
• Assignee: -
• Target version: -
• Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN

The build of a package with extensions, such as msgpack, is not reproducible due to temporary filenames appearing in the generated Makefile [1]:

grep gem lib/ruby/gems/2.5.0/gems/msgpack-1.2.4/ext/msgpack/Makefile
sitearchdir = $(DESTDIR)./.gem.20181114-6835-czlel4
sitelibdir = $(DESTDIR)./.gem.20181114-6835-czlel4

This issue comes from the creation of temporary files: the name is different across builds. More precisely, I've identified the following lines

• https://github.com/ruby/ruby/blob/trunk/lib/rubygems/ext/ext_conf_builder.rb#L16
• https://github.com/ruby/ruby/blob/trunk/lib/rubygems/ext/ext_conf_builder.rb#L29

There are several choices to fix this:

• use deterministic temporary filename: is there a lib to do this?
• provide these filenames as arguments and the use actual logic if these filenames are not provided: how to provide such filename?
• remove Makefile and gem_make.out files: I don't know how they are useful
• surely some other way to fix this...

I'm not a ruby programmer so I don't really know how it is difficult to implement these solutions.

What do you think about this?

[1] https://tests.reproducible-builds.org/archlinux/community/ruby-msgpack/ruby-msgpack-1.2.4-1-x86_64.pkg.tar.xz.html#ruby-msgpack--.-.----x--_--.pkg.tar---usr-lib-ruby-gems--.-.--gems-msgpack--.-.--ext-msgpack-Makefile

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Third Party's Issue

Why are such files packaged?

nobu (Nobuyoshi Nakada) wrote:

Why are such files packaged?

These files are written by gem intall in the directory specified by the --install-dir option. Distributions are taking this whole direcory to create the distribution package (.deb for instance).
I think the mkmf.log should not be written in the install-dir. But what about the Makefile? Could it be still used after the ruby gem build? If not, it could also be removed from the install-dir.

What do you think about removing these two files (Makefile and mkmf.log) from the --install-dir ?

[nobu (Nobuyoshi Nakada)]

lewo (lewo lewo) wrote:

I think the mkmf.log should not be written in the install-dir. But what about the Makefile? Could it be still used after the ruby gem build? If not, it could also be removed from the install-dir.

These (and page-Makefile.ri) are all garbages after installation.

What do you think about removing these two files (Makefile and mkmf.log) from the --install-dir ?

I dislike that rubygems installs junk files including *.o files.

nobu (Nobuyoshi Nakada) wrote:

These (and page-Makefile.ri) are all garbages after installation.

Hmm, I'm not sure to understand, because several distributions have these files remaining in the install-dir provided to rubygem, ie. these files are not garbage collected by rubygem. So, this is not what we are observing.
Or do you mean these files have to be removed by the distribution itself?

[nobu (Nobuyoshi Nakada)]

Regarding files under usr/​lib/​ruby/​gems/​2.​5.​0/​extensions/​x86_64-​linux/​2.​5.​0/​msgpack-​1.​2.​4/​, I think that rubygems is lazy not to select files properly.
And .​/​usr/​lib/​ruby/​gems/​2.​5.​0/​gems/​msgpack-​1.​2.​4/​ext/​msgpack/​ is a working directory where is not expected to be packaged, I think.
You may want to distclean there before packaging.

I created the PR https://github.com/rubygems/rubygems/pull/2481 to address these issues.