Bug #21995
open
mkmf generates a non-deterministic log file
Description
Hello,
When compiling Ruby's C extensions with mkmf, it can generate a mkmf.log file with non-deterministic content, especially at the line below:
ld: /tmp/rubytest.h8pay9/cc3fbuGm.o: in function `t':
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This can be problematic on some distributions/package-managers, since the mkmf.log file is included in the files generated by gem install.
To wrap up, I think the followings are desirable:
- Conventional file locations should be kept as is on the RubyGems side.
- Just deleting these files breaks compatibilities.
- These might be helpful when users report issues to gem owners with detailed information.
- It is helpful if generated file contents are as reproducible as possible.
- In some contexts where reproducibility matters, this issue makes packaging Ruby gems a bit harder.
- Luckily, workarounds are possible in distribution package recipes such as deleting these files.
- -> the component that generates
mkmf.logis mkmf, so it might be resolvable by modifying mkmf itself.
Thank you,
gemmaro.
Updated by gemmaro (Gemma Kosaka) about 1 month ago
I created a pull request about this.
Updated by hsbt (Hiroshi SHIBATA) 12 days ago
- Status changed from Open to Assigned
- Assignee set to hsbt (Hiroshi SHIBATA)
I will look this.
Updated by gemmaro (Gemma Kosaka) 12 days ago
Thanks for the reply! I'm sorry -- this pull request #note-1 currently fails CI, and I'm still investigating why. It's not ready for review yet, so it would be appreciated if you could revisit when it is.
Updated by Hanmac (Hans Mackowiak) 12 days ago
i think the main question is:
Why is mkmf.log file is included in the files generated by gem install?
Shouldn't it be okay to skip these files?
Especially for distributions/package-managers?
Updated by nobu (Nobuyoshi Nakada) 7 days ago
- Status changed from Assigned to Rejected
mkmf.log is not intended to be installed.
Updated by hsbt (Hiroshi SHIBATA) 7 days ago
- Status changed from Rejected to Assigned
Could you clarify what you mean by "distributions/package-managers"? Also, do they actually include the RubyGems log files (like mkmf.log) in their packages?
Updated by gemmaro (Gemma Kosaka) 7 days ago
Could you clarify what you mean by "distributions/package-managers"?
"distributions/package-managers" in the issue description refers, broadly, to systems that allow software packages to cooperate -- automatically tracking dependencies, locating files, and managing environment variables -- while making them easily installable, upgradable, and removable. Here, however, I specifically mean systems with stricter reproducibility guarantees, such as the Guix or Nix package managers.
For those unfamiliar with such systems, these are conceptually similar to Bundler. For example, running guix shell ruby ruby-nokogiri drops you into a shell with Ruby and Nokogiri installed and the environment fully set up. The key difference is that these guarantee that installing the same package on the same architecture always produces bit-for-bit identical results.
Also, do they actually include the RubyGems log files (like mkmf.log) in their packages?
Yes, at least with Guix, some packages do include the log files.
results of guix locate command
That said, as these files are increasingly being identified as a source of reproducibility check failures, existing packages may also start applying workarounds in their recipes, such as explicitly deleting these log files.
As I described in the issue, based on my research into the history of this topic, I had assumed that including log files such as mkmf.log in the files generated by gem install was intended behavior. That said, if gem install were changed to skip these files, I would consider that a step in the right direction (for reproducibility-aware package managers).