Bug #18000
closedhave_library doesn't work when ruby is compiled with --disable-shared --disable-install-static-library
Description
Related [Feature #12845]
If you compile ruby with --disable-shared --disable-install-static-library
, then many C-extension won't compile anymore. For instance RedCloth
# extconf.rb
require 'mkmf'
CONFIG['warnflags'].gsub!(/-Wshorten-64-to-32/, '') if CONFIG['warnflags']
$CFLAGS << ' -O0 -Wall ' if CONFIG['CC'] =~ /gcc/
dir_config("redcloth_scan")
have_library("c", "main")
create_makefile("redcloth_scan")
#mkmf.log
"gcc -o conftest -I/usr/local/include/ruby-3.1.0/x86_64-linux -I/usr/local/include/ruby-3.1.0/ruby/backward -I/usr/local/include/ruby-3.1.0 -I. -I/usr/local/include -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wduplicated-cond -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wwrite-strings -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-compat -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute=format -Wsuggest-attribute=noreturn -Wunused-variable -Wundef -O0 -Wall conftest.c -L. -L/usr/local/lib -Wl,-rpath,/usr/local/lib -L. -L/usr/local/lib -fstack-protector-strong -rdynamic -Wl,-export-dynamic -Wl,-rpath,/usr/local/lib -L/usr/local/lib -lruby-static -lz -lpthread -lrt -lrt -lgmp -ldl -lcrypt -lm -lm -lc"
/usr/bin/ld: cannot find -lruby-static
collect2: error: ld returned 1 exit status
checked program was:
/* begin */
1: #include "ruby.h"
2:
3: int main(int argc, char **argv)
4: {
5: return !!argv[argc];
6: }
/* end */
We'd like to use both flags, the first because it provide a small performance improvement, the second because libruby-static.a
is enormous (up to 120MiB on debug builds).
@alanwu (Alan Wu) says it's theoretically possible to compile with just the headers.
Files
Updated by xtkoba (Tee KOBAYASHI) almost 3 years ago
A workaround would be to place a dummy libruby-static.a
wherever ld
can find it.
!<arch>
Updated by byroot (Jean Boussier) almost 3 years ago
Thanks for the workaround, it does work for that very specific gem, however gems checking for specific Ruby APIs are broken by it. e.g. stackprof
require 'mkmf'
if have_func('rb_postponed_job_register_one') &&
have_func('rb_profile_frames') &&
have_func('rb_tracepoint_new') &&
have_const('RUBY_INTERNAL_EVENT_NEWOBJ')
create_makefile('stackprof/stackprof')
else
fail 'missing API: are you using ruby 2.1+?'
end
checking for rb_postponed_job_register_one()... no
*** extconf.rb failed ***
"gcc -o conftest -I/usr/local/include/ruby-3.1.0/x86_64-linux -I/usr/local/include/ruby-3.1.0/ruby/backward -I/usr/local/include/ruby-3.1.0 -I. -I/usr/local/include -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wduplicated-cond -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wwrite-strings -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-compat -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute=format -Wsuggest-attribute=noreturn -Wunused-variable -Wundef conftest.c -L. -L/usr/local/lib -Wl,-rpath,/usr/local/lib -L. -L/usr/local/lib -fstack-protector-strong -rdynamic -Wl,-export-dynamic -Wl,-rpath,/usr/local/lib -L/usr/local/lib -lruby-static -lz -lpthread -lrt -lrt -lgmp -ldl -lcrypt -lm -lm -lc"
conftest.c: In function ‘t’:
conftest.c:14:57: error: ‘rb_postponed_job_register_one’ undeclared (first use in this function)
14 | int t(void) { void ((*volatile p)()); p = (void ((*)()))rb_postponed_job_register_one; return !p; }
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
conftest.c:14:57: note: each undeclared identifier is reported only once for each function it appears in
conftest.c: At top level:
cc1: warning: unrecognized command line option ‘-Wno-self-assign’
cc1: warning: unrecognized command line option ‘-Wno-parentheses-equality’
cc1: warning: unrecognized command line option ‘-Wno-constant-logical-operand’
checked program was:
/* begin */
1: #include "ruby.h"
2:
3: /*top*/
4: extern int t(void);
5: int main(int argc, char **argv)
6: {
7: if (argc > 1000000) {
8: int (* volatile tp)(void)=(int (*)(void))&t;
9: printf("%d", (*tp)());
10: }
11:
12: return !!argv[argc];
13: }
14: int t(void) { void ((*volatile p)()); p = (void ((*)()))rb_postponed_job_register_one; return !p; }
/* end */
"gcc -o conftest -I/usr/local/include/ruby-3.1.0/x86_64-linux -I/usr/local/include/ruby-3.1.0/ruby/backward -I/usr/local/include/ruby-3.1.0 -I. -I/usr/local/include -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wduplicated-cond -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wwrite-strings -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-compat -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute=format -Wsuggest-attribute=noreturn -Wunused-variable -Wundef conftest.c -L. -L/usr/local/lib -Wl,-rpath,/usr/local/lib -L. -L/usr/local/lib -fstack-protector-strong -rdynamic -Wl,-export-dynamic -Wl,-rpath,/usr/local/lib -L/usr/local/lib -lruby-static -lz -lpthread -lrt -lrt -lgmp -ldl -lcrypt -lm -lm -lc"
/usr/bin/ld: /tmp/ccLbP8Oe.o: in function `t':
/usr/local/lib/ruby/gems/3.1.0/gems/stackprof-0.2.17/ext/stackprof/conftest.c:15: undefined reference to `rb_postponed_job_register_one'
collect2: error: ld returned 1 exit status
checked program was:
/* begin */
1: #include "ruby.h"
2:
3: /*top*/
4: extern int t(void);
5: int main(int argc, char **argv)
6: {
7: if (argc > 1000000) {
8: int (* volatile tp)(void)=(int (*)(void))&t;
9: printf("%d", (*tp)());
10: }
11:
12: return !!argv[argc];
13: }
14: extern void rb_postponed_job_register_one();
15: int t(void) { rb_postponed_job_register_one(); return 0; }
/* end */
Updated by jeremyevans0 (Jeremy Evans) over 2 years ago
- File no-static-no-shared.diff no-static-no-shared.diff added
- File rb_prefix_hack.diff rb_prefix_hack.diff added
While it is possible to compile with only the headers, that only works if you know what the headers should contain. If you aren't sure what the headers contain, and you have to test, you really need to test both compiling and linking to be reasonably sure whether the headers support the function in question. You can only do such testing if you have either the shared or static library available.
The issue here is how the mkmf have_func
method works. Currently, it writes a small C program and tries to compile and link it. If it cannot do so correctly, it determines the function doesn't work. You can work around this by only trying to compile and not link. However, that breaks cases that will compile but won't link, or won't compile unless you link. Such cases exist in the openssl extension, at least in my environment. Either I couldn't get Ruby to build with the changes, or the built Ruby wouldn't have a working openssl extension (needed for gem install
). I've attached my attempt at this in case you want to keep hacking to make it work. I don't think the approach is feasible, though.
One possible hack that may work for this use case is if have_func
is checking for a function that starts with rb_
, assume it is already defined. These calls are usually used to check that the Ruby version in use includes a function that wasn't defined in previous Ruby versions. If you are running the latest Ruby version, all of these functions are probably defined. This is obviously a hack we couldn't ship, though from my testing, it does allow gem install stackprof
. I've attached this hack as well.
I think the tool/mkconfig.rb
changes in both patches should be committed. These remove the -lruby-static
from LIBRUBYARG_STATIC
. After all, there is no point trying to link to a library that you know you didn't install. I've submitted a pull request for that: https://github.com/ruby/ruby/pull/4736
Updated by nobu (Nobuyoshi Nakada) over 2 years ago
We should disallow that combination, I think.
Otherwise, should drop also the headers.
Updated by jeremyevans0 (Jeremy Evans) over 2 years ago
nobu (Nobuyoshi Nakada) wrote in #note-4:
We should disallow that combination, I think.
Otherwise, should drop also the headers.
Here's a pull request to disallow the combination of --disable-shared --disable-install-static-library
: https://github.com/ruby/ruby/pull/4737. Not sure how @byroot (Jean Boussier) or @alanwu (Alan Wu) will feel about this approach, though.
Updated by byroot (Jean Boussier) over 2 years ago
Not sure how byroot (Jean Boussier) or alanwu (Alan Wu) will feel about this approach, though.
At the very least I'd say it's progress as my attempt to ship ruby builds with this combinations of flags would have immediately failed during the build rather than later in production.
But yeah I wish I could --disable-shared
without having to ship a huge 100MiB+ static library, but since I don't quite understand what it would take to support this combination of flags, I have no strong opinion about it.
Updated by nobu (Nobuyoshi Nakada) over 2 years ago
byroot (Jean Boussier) wrote in #note-6:
But yeah I wish I could
--disable-shared
without having to ship a huge 100MiB+ static library, but since I don't quite understand what it would take to support this combination of flags, I have no strong opinion about it.
With that combination, shouldn't we drop the header files too?
When both libraries are not installed, all undefined symbols need to be ignored or dynamic lookup, then have_func
and find_library
can't work as expected.
Updated by alanwu (Alan Wu) over 2 years ago
No objection from me to drop support for the --disable-shared --disable-install-static-library
config. I agree with Jean that it's an
improvement to give a clear message and fail fast.
With that combination, shouldn't we drop the header files too?
If we are keen on supporting that config, it should be possible to add
special handling to have_func
and other mkmf
methods to work without
libruby. On glibc platforms at least, this could be done by building the test
program as a shared library, and then using dlopen(3) with RTLD_NOW
to check
if every symbol resolves. The test program would still need the headers.
This testing process is similar to how Ruby would require the extension
once it's built, so maybe in some ways it's also a more accurate test.
It's already possible today to build an extension without libruby using
only the headers. Here's a small demo for it:
# A small demo for building an extension against a Ruby built with
# --disable-shared without libruby.
require 'rbconfig'
config = RbConfig::CONFIG
hdr_dir = config['rubyhdrdir']
arch_hdr_dir = config['rubyarchhdrdir']
# Note how this doesn't link against libruby at all
compile_command = "cc -shared -fpic -o ext.so -I #{hdr_dir} -I #{arch_hdr_dir} ext.c"
File.write("ext.c", <<~EOM)
#include <ruby/ruby.h>
void
Init_ext(void)
{
// Using two Ruby symbols. The static Ruby executable exports these.
rb_raise(rb_eRuntimeError, "hello from ext");
}
EOM
puts " Enabled shared: #{config['ENABLE_SHARED']}"
puts "Compile command: #{compile_command}"
system(compile_command)
require_relative 'ext'
Adding support takes more effort than rejecting the config outright of
course. Another easy option is making mkmf
error on that config.
We could still leave the headers for people that want to build
extensions without mkmf
. Maybe there are users writing extensions
in languages not supported by mkmf
.
Updated by jeremyevans (Jeremy Evans) over 2 years ago
- Status changed from Open to Closed
Applied in changeset git|06c3e80611b81ec8f251957328486e9b6dd18d3b.
Do not allow configuration where neither static or shared library is installed
Fixes [Bug #18000]