Misc #19758: Statically link ext/json

• Status: Assigned
• Assignee: nobu (Nobuyoshi Nakada)

[MyCo (Maik Menz)]

Hi,

I'm building Ruby both as dynamic and static library with MSVC for a project. Everything appears to work fine, but now I'm trying to use the json ext, and it only works with the dynamically linked version.
In the statically linked version it says it's missing json/pure but on closer inspection the reason it says that is because it can't find json/ext/parser (and probably also json/ext/generator) in the first place.

I can see that both parser & generator created static libs in the build directory but they aren't linked into the ruby lib.
With my limited knowledge of the building processes, my first attempt was to add both of those libs into LOCAL_LIBS and now they apear in the linking process.
But this still doesn't change anything. It's still not finding those 2 libs in the statically linked Ruby build.

What am I missing? What do I have to do, to get those linked into the static lib?

Regards
Maik

[MyCo (Maik Menz)]

EDIT: I'm using --enable-install-static-library --with-static-linked-ext and it appears as if non of the ext's are actually linked into the static lib.

[nobu (Nobuyoshi Nakada)]

Does generated ext/extinit.c include init(Init_parser, "json/ext/parser") line?

[MyCo (Maik Menz)]

nobu (Nobuyoshi Nakada) wrote in #note-2:

Does generated ext/extinit.c include init(Init_parser, "json/ext/parser") line?

Yes, it also contains the other libraries that are missing,eg. Init_socket

[MyCo (Maik Menz)]

The problem appears to be here:

$(LIBRUBY_A):	$(OBJS) $(INITOBJS)
		$(ECHO) linking static-library $(@:\=/)
		$(Q) $(AR) $(ARFLAGS)$@ $(OBJS) $(INITOBJS)

where:

INITOBJS      = dmyext.$(OBJEXT) dmyenc.$(OBJEXT)

This should actually point to ext/extinit.$(OBJEXT) enc/encinit.$(OBJEXT) but no matter what I try, it's always creating build issues. For example No such file or directory @ rb_sysopen - ext/extinit.c, maybe someone can give me a hint what's going on. I don't understand how this file doesn't exist even if it's a dependency of a build step.

[mame (Yusuke Endoh)]

• Assignee set to nobu (Nobuyoshi Nakada)

[jmarrec (Julien Marrec)]

Hello,

I am facing the same issue today, though I am presently on Ubuntu, not windows. I realized that Init_ext was doing nothing (I couldn't then use require 'bigdecimal' without a crash) by using a debugger and I noticed that it was picking up the empty one from dmyext.c and not the auto-generated ext/extinit.c one, and I identified this was because tof the INITOBJS in that AR call, which lead me to this issue. I have specifically uncommented all lines in ext/Setup (except the win32 ones) to have option nodynamic enabled in particular.

I tried specifically not linking to INITOBJS by patching Makefile.in as follow, but I get a failure to build. At that point in time when the libruby-static.a target is called, the EXTOBJS / ENCOBJS do not exist on disk yet (EXTOBJS=ext/extinit.o, ENCOBJS=enc/encinit.o enc/libenc.a enc/libtrans.a).

And libruby-static.a is needed for miniruby (I think), which is needed to build those EXT/ENC objects. So there needs to be some kind of final pass with AR overriding the Init_ext/Init_enc or something? And perhaps also merging the inidividual ext/ && enc .a libraries into the libruby-static.a itself so everything is self-contained?

Do you have any tips please?

---

(Failed) things I tried:

I'm not experienced enough with the ruby build system (I have trouble following how the extensions's target are called) to make that happen, so in the interim I tried to manually fix the libruby-static.a via new make target that calls ar to replace INITOBJS with EXTOBJS / ENCOBJS but that failed too, because it's missing the individual Init_ (eg: Init_digest, Init_big5, etc).

Truncated failure when make && make final-libruby_a && verify-static-library:

/usr/bin/ld: libruby-static.a(extinit.o): in function `Init_ext':
/home/julien/Software/Others/ruby-download/ruby-3.2.2-build/ext/extinit.c:12: undefined reference to `Init_bigdecimal'
/usr/bin/ld: /home/julien/Software/Others/ruby-download/ruby-3.2.2-build/ext/extinit.c:13: undefined reference to `Init_escape'
/usr/bin/ld: /home/julien/Software/Others/ruby-download/ruby-3.2.2-build/ext/extinit.c:14: undefined reference to `Init_continuation

Bad patch I tried to apply:

diff --git a/template/Makefile.in b/template/Makefile.in
--- a/template/Makefile.in
+++ b/template/Makefile.in
@@ -333,7 +333,18 @@ $(LIBRUBY_A):
 		@$(RM) $@
 		@-[ -z "$(EXTSTATIC)" ] || $(PRE_LIBRUBY_UPDATE)
 		$(ECHO) linking static-library $@
-		$(Q) $(AR) $(ARFLAGS) $@ $(LIBRUBY_A_OBJS) $(INITOBJS)
+		$(ECHO) LIBRUBY_A_OBJS=$(LIBRUBY_A_OBJS)
+		$(ECHO) INITOBJS=$(INITOBJS)
+		$(ECHO) EXTOBJS=$(EXTOBJS)
+		$(ECHO) ENCOBJS=$(ENCOBJS)
+		$(ECHO) ext/extinit.$(OBJEXT) enc/encinit.$(OBJEXT)
+		$(Q) if [ -z "$(EXTSTATIC)" ]; then \
+		  $(ECHO0) CMD=$(AR) $(ARFLAGS) $@ $(LIBRUBY_A_OBJS) $(INITOBJS); \
+		  $(AR) $(ARFLAGS) $@ $(LIBRUBY_A_OBJS) $(INITOBJS); \
+		else \
+		  $(ECHO0) CMD=$(AR) $(ARFLAGS) $@ $(LIBRUBY_A_OBJS) $(INITOBJS); \
+		  $(AR) $(ARFLAGS) $@ $(LIBRUBY_A_OBJS) $(INITOBJS); \
+		fi
 		$(Q) if [ 'no' != '$(YJIT_SUPPORT)' ]; then \
 		  set -eu$(V0:1=x) && \
 		  $(ECHO0) 'merging $(YJIT_LIBS) into $@' && \
@@ -346,7 +357,18 @@ $(LIBRUBY_A):
 		fi
 		@-$(RANLIB) $@ 2> /dev/null || true
 
-verify-static-library: $(LIBRUBY_A)
+final-libruby_a:
+		@-[ -z "$(EXTSTATIC)" ]
+		$(ECHO) Replacing INCOBJS with EXTOBJS and ENCOBS in static-library $(LIBRUBY_A)
+		$(ECHO) LIBRUBY_A_OBJS=$(LIBRUBY_A_OBJS)
+		$(ECHO) INITOBJS=$(INITOBJS)
+		$(ECHO) EXTOBJS=$(EXTOBJS)
+		$(ECHO) ENCOBJS=$(ENCOBJS)
+		$(AR) dD $(LIBRUBY_A) $(INITOBJS)
+		$(AR) rD $(LIBRUBY_A) $(EXTOBJS) $(ENCOBJS)
+		@-$(RANLIB) $(LIBRUBY_A) 2> /dev/null || true
+
+verify-static-library:
 		$(ECHO) verifying static-library $@
 		@$(PURIFY) $(CC) $(EXE_LDFLAGS) $(XLDFLAGS) $(MAINOBJ) $(LIBRUBY_A) $(MAINLIBS) $(EXTLIBS) $(LIBS) $(OUTFLAG)conftest$(EXEEXT)
 		@$(RMALL) conftest$(EXEEXT) conftest.c conftest.dSYM

My configure call is as follows:

../ruby-patching/configure    --prefix=/ '--bindir=${prefix}/bin' '--sbindir=${prefix}/bin' '--libdir=${prefix}/lib' '--includedir=${prefix}/include' '--oldincludedir=${prefix}/include'    --disable-shared --enable-static    --disable-install-doc --with-static-linked-ext --enable-load-relative    --with-zlib-dir=/home/julien/.conan2/p/b/zlib1b2fc466a3731/p    --with-openssl-dir=/home/julien/.conan2/p/b/opensccd63d9154ad3/p    --with-libffi-dir=/home/julien/.conan2/p/b/libffe070527d1c02e/p    --with-libyaml-dir=/home/julien/.conan2/p/b/libyae7bed9b8d84ed/p    --with-readline-dir=/home/julien/.conan2/p/b/readl6fb13f62b9c29/p    --with-gmp-dir=/home/julien/.conan2/p/b/gmp598d198a78496/p    --with-opt-dir=/home/julien/.conan2/p/b/zlib1b2fc466a3731/p:/home/julien/.conan2/p/b/opensccd63d9154ad3/p:/home/julien/.conan2/p/b/libffe070527d1c02e/p:/home/julien/.conan2/p/b/libyae7bed9b8d84ed/p:/home/julien/.conan2/p/b/readl6fb13f62b9c29/p:/home/julien/.conan2/p/b/gmp598d198a78496/p    --disable-jit-support     --enable-mkmf-verbose    --enable-debug-env

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Assigned