Misc #20240: Unable to build ruby 3.1.0 on macOS when shared due to dylibs (libgmp) not found when running miniruby

• Status: Open
• Assignee: -

[jmarrec (Julien Marrec)]

I am trying to develop a conan (the C/C++ package manager) recipe for Ruby. The recipe would allow downstream users to 1) get a runnable ruby executable, and 2) be able to link to ruby, or embbed it in a C/C++ program if built statically, in an easy way.

Currently there is an existing ruby 3.1.0 recipe that I'm trying to adapt, so I have to support this version.

First off, let me say that I can succesfully build with 3.3.0, so I know something has changed for the better since then. I'm just at a lost when figuring out what I need to backport to make 3.1.0 work.

The original issue is that it appears miniruby is looking for some dylibs and not finding them. Even if I do define LD_LIBRARY_PATH, DYLD_LIBRARY_PATH or DYLD_FALLBACK_LIBRARY_PATH (any combinations of these three) in my env.

dsymutil exe/ruby; { test -z '' || codesign -s '' -f exe/ruby; }
./miniruby \
	    -e 'prog, dest, inst = ARGV; dest += "/ruby"' \
	    -e 'exit unless prog==inst' \
	    -e 'unless prog=="ruby"' \
	    -e '  begin File.unlink(dest); rescue Errno::ENOENT; end' \
	    -e '  File.symlink(prog, dest)' \
	    -e 'end' \
	ruby exe ruby
dyld[59344]: Library not loaded: @rpath/libgmp.10.dylib
  Referenced from: <356E0011-6223-321A-9179-D55618D248D0> /Users/julien/.conan2/p/b/ruby9cafa28a7060d/b/build-release/miniruby
  Reason: no LC_RPATH's found
make: *** [exe/ruby] Abort trap: 6
make: *** Deleting file `exe/ruby'

It seems that something is unsetting the variables, because this for eg works fine

DYLD_LIBRARY_PATH=/Users/julien/.conan2/p/b/zlib1f8e7d96319f0/p/lib:/Users/julien/.conan2/p/b/opense854e464e8ff6/p/lib:/Users/julien/.conan2/p/b/libyae2f0aa15c9e92/p/lib:/Users/julien/.conan2/p/b/libff05fe9d5b96f79/p/lib:/Users/julien/.conan2/p/b/readl0d0041a63fa03/p/lib:/Users/julien/.conan2/p/b/termc22b5bb1515971/p/lib:/Users/julien/.conan2/p/b/gmp676fa41eaa3d6/p/lib: /Users/julien/.conan2/p/b/ruby9cafa28a7060d/b/build-release/miniruby -e "puts 'Hello, world'"

My configure call is like this:

./configure --enable-shared --disable-static --prefix=/ '--bindir=${prefix}/bin' '--sbindir=${prefix}/bin' '--libdir=${prefix}/lib' '--includedir=${prefix}/include' '--oldincludedir=${prefix}/include' --disable-install-doc --enable-load-relative --with-zlib-dir=/Users/julien/.conan2/p/b/zlib1f8e7d96319f0/p --with-openssl-dir=/Users/julien/.conan2/p/b/opense854e464e8ff6/p --with-libffi-dir=/Users/julien/.conan2/p/b/libff05fe9d5b96f79/p --with-libyaml-dir=/Users/julien/.conan2/p/b/libyae2f0aa15c9e92/p --with-readline-dir=/Users/julien/.conan2/p/b/readl0d0041a63fa03/p --with-gmp-dir=/Users/julien/.conan2/p/b/gmp676fa41eaa3d6/p --with-opt-dir=/Users/julien/.conan2/p/b/opense854e464e8ff6/p:/Users/julien/.conan2/p/b/libff05fe9d5b96f79/p:/Users/julien/.conan2/p/b/libyae2f0aa15c9e92/p:/Users/julien/.conan2/p/b/readl0d0041a63fa03/p:/Users/julien/.conan2/p/b/gmp676fa41eaa3d6/p --disable-jit-support

I have tried to backport https://github.com/ruby/ruby/pull/6296/files and https://github.com/ruby/ruby/commit/48644e71096c70132be9dfdcbfb414ec2e68d18b and https://github.com/ruby/ruby/pull/8730 amongst other things but I can't make it work. (I even tried a more brute force approach patching a lot of files by diffing 3.3.0 with 3.1.0, but please note I don't know what I'm doing... and I can get to the install step but then I get some errors about Psych / libymal and undefined Gem::Install:Zlib).

I would greatly appreciate if someone can spare some time to help me wrap this up (I've been trying to make the recipe work for so long that I'm about to give up...)

[alanwu (Alan Wu)]

You could try building using --without-gmp to sidestep this problem.

[jmarrec (Julien Marrec)]

Thank you for your reponse!

Ideally I'd like to support it.

I've tried it, I get similar issues for some of the other dylibs, during make install. Psych is built correctly, just can't be installed. Zlib looks to be the same issue.

ruby/3.1.0: Calling package()
ruby/3.1.0: RUN: make install DESTDIR=/Users/julien/.conan2/p/b/rubyc85244d60c2fd/p -j10
Downloading bundled gem rbs-3.1.0...
Downloading bundled gem debug-1.6.3...
	BASERUBY = /usr/bin/ruby --disable=gems
	CC = clang
	LD = clang
	LDSHARED = clang -dynamiclib
	CFLAGS = -fPIC -O3 -fPIC -O3 -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wdivision-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wextra-tokens -Wundef -std=gnu99 -fno-common -pipe
	XCFLAGS = -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -fvisibility=hidden -DRUBY_EXPORT -I. -I.ext/include/arm64-darwin23 -I/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/include -I/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src -I/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/enc/unicode/13.0.0
	CPPFLAGS = -DNDEBUG -DNDEBUG -I/Users/julien/.conan2/p/b/zlib1f8e7d96319f0/p/include -I/Users/julien/.conan2/p/b/opense854e464e8ff6/p/include -I/Users/julien/.conan2/p/b/libff05fe9d5b96f79/p/include -I/Users/julien/.conan2/p/b/libyae2f0aa15c9e92/p/include -I/Users/julien/.conan2/p/b/readl0d0041a63fa03/p/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT
	DLDFLAGS = -Wl,-undefined,dynamic_lookup -L/Users/julien/.conan2/p/b/zlib1f8e7d96319f0/p/lib -L/Users/julien/.conan2/p/b/opense854e464e8ff6/p/lib -L/Users/julien/.conan2/p/b/libff05fe9d5b96f79/p/lib -L/Users/julien/.conan2/p/b/libyae2f0aa15c9e92/p/lib -L/Users/julien/.conan2/p/b/readl0d0041a63fa03/p/lib -install_name @executable_path/../lib/libruby.3.1.dylib -compatibility_version 3.1 -current_version 3.1.0  -fstack-protector-strong -framework CoreFoundation  -fstack-protector-strong -framework CoreFoundation
	SOLIBS = -lpthread -ldl -lobjc
	LANG = en_US.UTF-8
	LC_ALL = en_US.UTF-8
	LC_CTYPE =
	MFLAGS = - --jobserver-fds=4,5 -j
Apple clang version 15.0.0 (clang-1500.1.0.2.5)
Target: arm64-apple-darwin23.2.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
generating arm64-darwin23-fake.rb
generating enc.mk
arm64-darwin23-fake.rb updated
making srcs under enc
making enc
make[1]: Nothing to be done for `enc'.
make[1]: Nothing to be done for `srcs'.
generating transdb.h
Extracting bundle gem debug-1.6.3...
transdb.h unchanged
generating makefiles ext/configure-ext.mk
making trans
make[1]: Nothing to be done for `/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/enc/trans'.
making encs
ext/configure-ext.mk updated
Extracting bundle gem rbs-3.1.0...
make[1]: Nothing to be done for `encs'.
Unpacked gems/debug-1.6.3.gem
generating makefile exts.mk
exts.mk unchanged
Unpacked gems/rbs-3.1.0.gem
/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/revision.h unchanged
make[1]: Nothing to be done for `note'.
./miniruby -I/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib -I. -I.ext/common  /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/runruby.rb --extout=.ext  -- --disable-gems -r./arm64-darwin23-fake /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb --make="/Library/Developer/CommandLineTools/usr/bin/make" --dest-dir="/Users/julien/.conan2/p/b/rubyc85244d60c2fd/p" --extout=".ext" --ext-build-dir="./ext" --mflags="- --jobserver-fds=4,5 -j" --make-flags=" --jobserver-fds=4,5 -j -- DESTDIR=/Users/julien/.conan2/p/b/rubyc85244d60c2fd/p" --data-mode=0644 --prog-mode=0755 --installed-list .installed.list --mantype="doc"  --gnumake --install=all --exclude=doc
installing binary commands:         //bin
installing base libraries:          //lib
installing arch files:              //lib/ruby/3.1.0/arm64-darwin23
installing pkgconfig data:          //lib/pkgconfig
installing extension objects:       //lib/ruby/3.1.0/arm64-darwin23
installing extension objects:       //lib/ruby/site_ruby/3.1.0/arm64-darwin23
installing extension objects:       //lib/ruby/vendor_ruby/3.1.0/arm64-darwin23
installing extension headers:       //include/ruby-3.1.0/arm64-darwin23
installing extension scripts:       //lib/ruby/3.1.0
installing extension scripts:       //lib/ruby/site_ruby/3.1.0
installing extension scripts:       //lib/ruby/vendor_ruby/3.1.0
installing extension headers:       //include/ruby-3.1.0/ruby
installing command scripts:         //bin
installing library scripts:         //lib/ruby/3.1.0
installing common headers:          //include/ruby-3.1.0
installing manpages:                //share/man/man1
installing default gems from lib:   //lib/ruby/gems/3.1.0
                                    abbrev 0.1.0
                                    base64 0.1.1
                                    benchmark 0.2.0
                                    bundler 2.3.3
/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/yaml.rb:3: warning: It seems your ruby installation is missing psych (for YAML output).
To eliminate this warning, please install libyaml and reinstall your ruby.
/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/installer.rb:343:in `rescue in install': uninitialized constant Gem::Installer::Zlib (NameError)
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/installer.rb:293:in `install'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:899:in `block in install'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:713:in `no_write'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:899:in `install'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1002:in `block in install_default_gem'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:989:in `each'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:989:in `install_default_gem'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:927:in `block in <main>'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1127:in `block in <main>'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1124:in `each'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1124:in `<main>'
<internal:/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/core_ext/kernel_require.rb>:85:in `require': dlopen(/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/build-release/.ext/arm64-darwin23/psych.bundle, 0x0009): Library not loaded: @rpath/libyaml.dylib (LoadError)
  Referenced from: <C2079A68-F56A-3359-8762-39D3F1940B29> /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/build-release/.ext/arm64-darwin23/psych.bundle
  Reason: tried: '/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/build-release/libyaml.dylib' (no such file), '/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/build-release/libyaml.dylib' (no such file) - /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/build-release/.ext/arm64-darwin23/psych.bundle
	from <internal:/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/build-release/.ext/common/psych.rb:13:in `<top (required)>'
	from <internal:/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/yaml.rb:4:in `<top (required)>'
	from <internal:/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems.rb:619:in `load_yaml'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/config_file.rb:346:in `load_file'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/config_file.rb:189:in `initialize'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems.rb:331:in `new'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems.rb:331:in `configuration'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/user_interaction.rb:167:in `verbose'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/installer.rb:544:in `generate_bin_script'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:905:in `generate_bin_script'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:873:in `generate_bin_script'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/installer.rb:506:in `block in generate_bin'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/installer.rb:482:in `each'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/installer.rb:482:in `generate_bin'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/lib/rubygems/installer.rb:323:in `install'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:899:in `block in install'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:713:in `no_write'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:899:in `install'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1002:in `block in install_default_gem'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:989:in `each'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:989:in `install_default_gem'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:927:in `block in <main>'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1127:in `block in <main>'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1124:in `each'
	from /Users/julien/.conan2/p/b/rubyc85244d60c2fd/b/src/tool/rbinstall.rb:1124:in `<main>'
make: *** [do-install-nodoc] Error 1