Misc #17337: Don't embed Ruby build-time configuration in Ruby

• Status: Open
• Assignee: -

[vo.x (Vit Ondruch)]

When Ruby 3.0 is built without C++ compiler available, subsequent builds of Eventmachine extensions (or any other gems that require C++ compiler) fail:

... snip ...

"make \"DESTDIR=\""
I. -I/usr/include -I/usr/include/ruby/backward -I/usr/include -I.  -DHAVE_OPENSSL_SSL_H -DHAVE_OPENSSL_ERR_H -DWITH_SSL -DBUILD_FOR_RUBY -DHAVE_RB_THREAD_CALL_WITHOUT_GVL -DHAVE_RB_THREAD_FD_SELECT -DHAVE_TYPE_RB_FDSET_T -DHAVE_RB_WAIT_FOR_SINGLE_FD -DHAVE_RB_TIME_NEW -DHAVE_INOTIFY_INIT -DHAVE_INOTIFY -DHAVE_WRITEV -DHAVE_PIPE2 -DHAVE_ACCEPT4 -DHAVE_CONST_SOCK_CLOEXEC -DOS_UNIX -DHAVE_EPOLL_CREATE -DHAVE_EPOLL -DHAVE_CLOCK_GETTIME -DHAVE_CONST_CLOCK_MONOTONIC_RAW -DHAVE_CONST_CLOCK_MONOTONIC    -fPIC -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection  -m64 -o binder.o -c binder.cpp
make: I.: No such file or directory
make: [Makefile:237: binder.o] Error 127 (ignored)

This happens since 1.

I would like to question not just the commit, but the whole principle. Availability of C++ compiler during build of Ruby does not have anything to do with availability of C++ compiler during build of whatever Ruby extension.

Moreover, the machine where Ruby is built might be different from the machine where Ruby is used and the extension is installed. Typical examples are binary Linux distributions. RVM also uses precompiled Ruby binaries if I am not mistaken.

Therefore, I would appreciate if we reconsider embedding Ruby build-time configuration in Ruby and reusing it for Ruby extension build. This would solve the issue of Eventmachine I started with, and also issues such as #14422

Just FTR, to avoid this kind of issues, as a Ruby maintainer on Red Hat platforms, I am considering to ship rbconfig.rb in vendorlib, which would dynamically fix these issues. E.g. set the CONFIG['CXX'] based on C++ compiler availability during extension build. But I'd like to avoid it, at least as a downstream modification.

[shyouhei (Shyouhei Urabe)]

This is a tough problem.

Ruby ships several hundreds of C/C++ header files (those under include/ in our distribution). 3rd party extension libraries are expected to include them. The problem is, there are header files which depend on runtime API/ABI detected by our configure script. For an example we choose one of our 3 distinct select(2) wrapper depending on how many file descriptor the system can take at once. If the header file does not agree with generated ruby binary whatever bad thing can happen. As for C++ compilers, we want to know the availability of std::nullptr_t. If there is that type we provide extra function overloads that prevent warnings around casts.

The particular C++ feature detection business, I guess, could be possibly delayed until people configure their extension library. C++ nowadays are largely header-only. But if you want to separate the entire environment between the core and the extensions, I honestly have no idea if that is even possible.

[sawa (Tsuyoshi Sawada)]

• Description updated (diff)

[sawa (Tsuyoshi Sawada)]

• Subject changed from Don't embed Ruby build time configuration into Ruby to Don't embed Ruby build-time configuration in Ruby

[nobu (Nobuyoshi Nakada)]

Should we move such detections (for other than C, not only C++) to mkmf.rb?

[shyouhei (Shyouhei Urabe)]

nobu (Nobuyoshi Nakada) wrote in #note-4:

Should we move such detections (for other than C, not only C++) to mkmf.rb?

JFYI because compilers tend to LTO these days, libruby can be a compiler-specific internal representation of the entire source code, not an archive of object files. If that is the case, people cannot use arbitrary C++ compiler. It must exactly match the C compiler used to generate libruby. Detection of such things could be done in mkmf.rb I guess, but not as easy as to “move” the current logic.

[nobu (Nobuyoshi Nakada)]

shyouhei (Shyouhei Urabe) wrote in #note-5:

nobu (Nobuyoshi Nakada) wrote in #note-4:

Should we move such detections (for other than C, not only C++) to mkmf.rb?

JFYI because compilers tend to LTO these days, libruby can be a compiler-specific internal representation of the entire source code, not an archive of object files. If that is the case, people cannot use arbitrary C++ compiler. It must exactly match the C compiler used to generate libruby. Detection of such things could be done in mkmf.rb I guess, but not as easy as to “move” the current logic.

IOW, mkmf rather should fail in such situation?
(I'm not sure how to check it.)

[vo.x (Vit Ondruch)]

shyouhei (Shyouhei Urabe) wrote in #note-1:
Thx for elaborating some of the reasons behind.

However, I don't think that embedding of the compile time options addresses any of these concerns. If they should be addressed, that would mean to do the same configuration checks once done for Ruby also for build of extensions and fail the build if they differs.

Also, I don't really know, what component is responsible for number of file descriptors, but on Linux, I'd assume this is Kernel. But if the value is changed in between kernel versions, then the compile time choice of select(2) might not be optimal.

[vo.x (Vit Ondruch)]

shyouhei (Shyouhei Urabe) wrote in #note-5:

JFYI because compilers tend to LTO these days, libruby can be a compiler-specific internal representation of the entire source code, not an archive of object files. If that is the case, people cannot use arbitrary C++ compiler. It must exactly match the C compiler used to generate libruby. Detection of such things could be done in mkmf.rb I guess, but not as easy as to “move” the current logic.

This sounds that this applies to statically linked Ruby, or does it apply to shared libruby as well? I care just about the later if that makes any difference.

[shyouhei (Shyouhei Urabe)]

vo.x (Vit Ondruch) wrote in #note-7:

However, I don't think that embedding of the compile time options addresses any of these concerns. If they should be addressed, that would mean to do the same configuration checks once done for Ruby also for build of extensions and fail the build if they differs.

Right. I'm not saying we are doing well. Rather I say there is a headache. Ruby had already assumed the exact same runtime environment for both core & extensions. You want to separate them. I understand your motivation. But... that is harder than it sounds.

Also, I don't really know, what component is responsible for number of file descriptors, but on Linux, I'd assume this is Kernel. But if the value is changed in between kernel versions, then the compile time choice of select(2) might not be optimal.

That would happen. We have experienced situations like for instance getrandom(2) system call already implemented in the Kernel, but not yet available in glibc. Luckily that did not affect extension libraries though.

vo.x (Vit Ondruch) wrote in #note-8:

shyouhei (Shyouhei Urabe) wrote in #note-5:

JFYI because compilers tend to LTO these days, libruby can be a compiler-specific internal representation of the entire source code, not an archive of object files. If that is the case, people cannot use arbitrary C++ compiler. It must exactly match the C compiler used to generate libruby. Detection of such things could be done in mkmf.rb I guess, but not as easy as to “move” the current logic.

This sounds that this applies to statically linked Ruby, or does it apply to shared libruby as well? I care just about the later if that makes any difference.

AFAIK no dynamic linker interfaces with compilers' LTO facilities.

[vo.x (Vit Ondruch)]

shyouhei (Shyouhei Urabe) wrote in #note-9:

Ruby had already assumed the exact same runtime environment for both core & extensions.

While considering everything what was said here, this is not surprise. Nevertheless, I am not sure this is wildly known and documented enough.

However, in Red Hat ecosystem, this was never true. Build of Ruby RPM always happens on different system then build of RubyGems RPM packages. The configuration of the runtime system is certainly different from the builders. The underlying Kernel can change including changing important defaults. Using Ruby in containers, there is not enough information about undrlying Kernel, one cannot even rely on kernel-headers, because they very likely don't match. Discussing #16694 with RH toolchain folks, I was even suggested to use CLang to build Ruby itself, while keep using GCC to build extensions.

Therefore, given the real world use cases, isn't it time to reconsider this? I understand I'm leaving a lot of technical "details" aside, but if I could come in this case to Eventmachine upstream and say "you should not rely on Ruby detected configuration, do your own detection for best results, because Ruby upstream acknowledges the build time and runtime environment might differ", I'd be certainly in better place.

[Eregon (Benoit Daloze)]

vo.x (Vit Ondruch) wrote in #note-10:

if I could come in this case to Eventmachine upstream and say "you should not rely on Ruby detected configuration, do your own detection for best results, because Ruby upstream acknowledges the build time and runtime environment might differ", I'd be certainly in better place.

I disagree with that, specifically, C/C++/native extensions should rely on configuration from Ruby libraries such as mkmf and RbConfig.
Now, those values could reflect the runtime system and not the build system, I would not mind that and it sounds better (but it might be quite difficult to achieve for CRuby).
But I really don't want any native extension to detect its own C/C++ compiler, flags, etc, that leads to many problems and technical debt, notably when trying to compile on TruffleRuby which actually ships its own LLVM toolchain and where RbConfig::CONFIG["CC/CXX"] should be used to compile (which just works fine when using mkmf and not trying to hack around it).

FWIW TruffleRuby detects most things at runtime, and having a given fixed toolchain avoids so many of these headaches, which e.g. makes it possible to have a single prebuilt Linux binary running on all distributions, with the exception that the openssl C extension is recompiled on the runtime system at installation time because libssl varies too much per platform.

[vo.x (Vit Ondruch)]

Eregon (Benoit Daloze) wrote in #note-11:

vo.x (Vit Ondruch) wrote in #note-10:

if I could come in this case to Eventmachine upstream and say "you should not rely on Ruby detected configuration, do your own detection for best results, because Ruby upstream acknowledges the build time and runtime environment might differ", I'd be certainly in better place.

I disagree with that, specifically, C/C++/native extensions should rely on configuration from Ruby libraries such as mkmf and RbConfig.

I don't think I have anything specifically against mkmf. It is fine by me if it can be utilized. However, reusing embedded values from RbConfig is not always the best idea.

FWIW TruffleRuby detects most things at runtime, and having a given fixed toolchain avoids so many of these headaches, which e.g. makes it possible to have a single prebuilt Linux binary running on all distributions, with the exception that the openssl C extension is recompiled on the runtime system at installation time because libssl varies too much per platform.

Given the above mentioned relation between Kernel and Ruby build configurations, that leads me to believe that TruffleRuby is doing fine for most of the cases, but there are probably some corner cases. And similar situation is probably on Red Hat platforms.

Also, believe me that if you are using Fedora/RHEL provided libraries, you are basically in the same situation as you described the situation of TruffleRuby. But since we still want our users allow to use gem install, then I have to care also about these situations.

[Rinkana (Max Berends)]

• File Dockerfile Dockerfile added

It seems that this issue also breaks building the sassc gem from version 2.1 onward.
Building it on the ruby 3.0-rc docker image (ruby:3.0-rc-slim-buster) gives me:

    current directory: /usr/local/bundle/gems/sassc-2.4.0/ext
/usr/local/bin/ruby -I /usr/local/lib/ruby/site_ruby/3.0.0 -r ./siteconf20201129-11-yfpi8m.rb extconf.rb
creating Makefile

current directory: /usr/local/bundle/gems/sassc-2.4.0/ext
make "DESTDIR=" clean

current directory: /usr/local/bundle/gems/sassc-2.4.0/ext
make "DESTDIR="
compiling ./libsass/src/ast.cpp
make: I.: Command not found
make: [Makefile:237: ast.o] Error 127 (ignored)

[....]

compiling ./libsass/src/values.cpp
make: I.: Command not found
make: [Makefile:237: values.o] Error 127 (ignored)
linking shared-object sassc/libsass.so
make: shared: Command not found
make: [Makefile:262: libsass.so] Error 127 (ignored)
strip: 'libsass.so': No such file
make: *** [Makefile:263: libsass.so] Error 1

make failed, exit code 2

Gem files will remain installed in /usr/local/bundle/gems/sassc-2.4.0 for inspection.
Results logged to /usr/local/bundle/extensions/x86_64-linux/3.0.0/sassc-2.4.0/gem_make.out

This is with just a clean Rails install (6.1.0-rc1) and no additional gems.
sassc version 2.0.1 builds fine. As does sassc 2.4.0 on the ruby 2.7.2 image (ruby:2.7-slim-buster).

Attached is my Dockerfile. It requires an entrypoint.sh that contains just:

#!/bin/bash
set -e
exec "$@"

So it seems to me that this change can have a big impact as my setup is not too uncommon (correct me if i'm wrong).

[shyouhei (Shyouhei Urabe)]

We need to discuss the "whole principle" part. But in the meamtine we could revert this particular C++ compiler detection business to help eventmachine & sassc. Thoughts?

[vo.x (Vit Ondruch)]

shyouhei (Shyouhei Urabe) wrote in #note-14:

But in the meamtine we could revert this particular C++ compiler detection business to help eventmachine & sassc. Thoughts?

+1

Should I open separate ticket for the revert and keep this ticket open for continuing the "whole principle" discussion?

[shyouhei (Shyouhei Urabe)]

• Status changed from Open to Closed

Applied in changeset git|ccc828f49956424b2548e32cb3bc3a78e16e207b.

---

configure.ac: avoid squashing CXX=g++

We are discussing this issue at [Bug #17337] but in the meantime, leave
this questionable autoconf glitch as-is to save sassc and eventmachine.

[shyouhei (Shyouhei Urabe)]

• Status changed from Closed to Open

vo.x (Vit Ondruch) wrote in #note-15:

Should I open separate ticket for the revert and keep this ticket open for continuing the "whole principle" discussion?

Don’t worry, I have reverted that part already. Use this ticket for the discussion.

[webzorg (Lasha Abulashvili)]

I was told about this bug on freenode#ruby

It maybe related to my recent problem with installing pg gem on ruby 2.7.2

gem_make.out - https://paste.ubuntu.com/p/s6YD8C5yqS/
mkmf.log - https://paste.ubuntu.com/p/BKTVvdkJt6/

according to rvm logs the binary was simply downloaded precompiled.
https://paste.ubuntu.com/p/H7r4zjmRTN/

[jeremyevans0 (Jeremy Evans)]

• Tracker changed from Bug to Misc
• ruby -v deleted (ruby 3.0.0dev (2020-11-04 master 704fb0b815) [x86_64-linux])
• Backport deleted (2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN)