Bug #16331: fails to build with BSD make when any -j option is given (including -j 1)

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: 2.6.5
• Backport: 2.5: REQUIRED, 2.6: REQUIRED, 2.7: DONE

[[e-mail] (Koichiro Iwao)]

When building Ruby parallely with BSD make (FreeBSD make), it fails to build.
Reported by FreeBSD Ruby folks. Please keep in mind I didn't find out the root cause.

OS: FreeBSD 12-STABLE
See also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=241633

How to reproduce

tar xf ruby-2.6.5.tar.gz
cd ruby-2.6.5
./configure --enable-shared --disable-readline --disable-libedit
make -j 8 # any value will cause build failure
(snip)
configuring -test-/st/numhash
configuring -test-/st/update
--- enc ---
ld: error: unable to find library -lruby
cc: error: linker command failed with exit code 1 (use -v to see invocation)
--- exts.mk ---
configuring -test-/string
configuring -test-/struct
configuring -test-/symbol
--- enc ---
*** [.ext/x86_64-freebsd12.1/enc/encdb.so] Error code 1

BSD make without -j option: success

cd ruby-2.6.5
make distclean
./configure --enable-shared --disable-readline --disable-libedit
make
`ruby' is up to date.
*** Following extensions are not compiled:
readline:
	Could not be configured. It will not be installed.
	/home/sair/tmp/ruby-2.6.5/ext/readline/extconf.rb:52: readline not found
	Check ext/readline/mkmf.log for more details.
gdbm:
	Could not be configured. It will not be installed.
	Check ext/gdbm/mkmf.log for more details.
*** Fix the problems, then remove these directories and try again if you want.
making enc
making trans
`trans' is up to date.
making encs
Generating RDoc documentation

Using GNU make: success

cd ruby-2.6.5
export MAKE=gmake # GNU make
make distclean
./configure --enable-shared --disable-readline --disable-libedit
$MAKE -j 8

[[e-mail] (Koichiro Iwao)]

One of the FreeBSD folks Yuichiro NAITO provided more detail.

See comment 13:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=241633#c13

[[e-mail] (Koichiro Iwao)]

• Subject changed from fails to build with BSD make possibly due to race condition to fails to build with BSD make when any -j option is given (including -j 1)

Now it turned out the build fails even -j 1 is specified. Any value to -j option will fail.
Not giving -j option goes fine.

Further information from Yuichiro NAITO:

最初にレースコンディションと言っていたのは誤りで
-j の有無で shell に渡すコマンドの範囲が異なるのが原因でした。
基本的にはビルド手順を1行づつ shell に渡すのが本来の make の実装なのですが、
BSD make は -j オプションをつけると、ビルド手順が複数行ある場合にまとめて shell に渡すようになります。
そうすると、途中で exec があった場合、残りのコマンドが実行されなくなります。
Ruby の Makefile では行の最後のコマンドに exec 使うことで、
最後の fork(2) 実行を省略しようという意図のように見えます。

結局のところ BSD make の互換性問題のように思います。

[[e-mail] (Koichiro Iwao)]

• Description updated (diff)

[naruse (Yui NARUSE)]

[e-mail] (Koichiro Iwao) wrote:

Now it turned out the build fails even -j 1 is specified. Any value to -j option will fail.
Not giving -j option goes fine.

Further information from Yuichiro NAITO:

最初にレースコンディションと言っていたのは誤りで
-j の有無で shell に渡すコマンドの範囲が異なるのが原因でした。
基本的にはビルド手順を1行づつ shell に渡すのが本来の make の実装なのですが、
BSD make は -j オプションをつけると、ビルド手順が複数行ある場合にまとめて shell に渡すようになります。
そうすると、途中で exec があった場合、残りのコマンドが実行されなくなります。
Ruby の Makefile では行の最後のコマンドに exec 使うことで、
最後の fork(2) 実行を省略しようという意図のように見えます。

結局のところ BSD make の互換性問題のように思います。

It sounds Ruby hits the issue described in make -B:
https://www.freebsd.org/cgi/man.cgi?query=make&apropos=0&sektion=0&manpath=FreeBSD+12.1-RELEASE+and+Ports&arch=default&format=html

[nobu (Nobuyoshi Nakada)]

[e-mail] (Koichiro Iwao) wrote:

One of the FreeBSD folks Yuichiro NAITO provided more detail.

See comment 13:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=241633#c13

The conditions in libruby-static and libruby-shared are different intentionally.

First:
It seems that it is not intended to call $(PRE_LIBRARY_UPDATE) in Ruby's Makefile,
when libruby.so.26 is built.
Comparing to libruby-static library build,
Ruby's Makefile should be fixed like Makefile.in.patch.

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed

Applied in changeset git|faa4f4f23fbb6b48b158cc38d9b509fd00838976.

---

Get rid of FreeBSD make incompatibility [Bug #16331]

FreeBSD make works differently with -j option.

-j max_jobs
Specify the maximum number of jobs that make may have running
at any one time. The value is saved in .MAKE.JOBS. Turns
compatibility mode off, unless the B flag is also specified.
When compatibility mode is off, all commands associated with a
target are executed in a single shell invocation as opposed to
the traditional one shell invocation per line. This can break
traditional scripts which change directories on each command
invocation and then expect to start with a fresh environment on
the next line. It is more efficient to correct the scripts
rather than turn backwards compatibility on.

Stop using exit, cd, exec in middle of commands.

[yasuhirokimura (Yasuhiro KIMURA)]

Are there plan to merge this fix to ruby_2_6 branch?

[nobu (Nobuyoshi Nakada)]

• Backport changed from 2.5: UNKNOWN, 2.6: UNKNOWN to 2.5: REQUIRED, 2.6: REQUIRED

[yasuhirokimura (Yasuhiro KIMURA)]

• File buildtest-with-bmake-and-j-option.log buildtest-with-bmake-and-j-option.log added

I executed following shell script on my FreeBSD test environment and got attached log file.

#!/bin/sh

LANG=C
(
    echo System Information:
    echo ----------------------------------------------------------------------
    dmesg
    echo ----------------------------------------------------------------------
    echo Package Information:
    echo ----------------------------------------------------------------------
    pkg info -aq
    echo ----------------------------------------------------------------------
    cd /tmp
    for ver in 2.6.5 2.7.0-rc2
    do
        echo Test build of ${ver}:
        tar xfpJ /net/freebsd/ports/distfiles/ruby/ruby-${ver}.tar.xz
        cd ruby-${ver}
        echo Configure:
        echo ----------------------------------------------------------------------
        ./configure
        echo ----------------------------------------------------------------------
        echo Build:
        echo ----------------------------------------------------------------------
        make -j $(sysctl -n kern.smp.cpus)
        echo ----------------------------------------------------------------------
        cd ..
        rm -rf ruby-${ver}
    done
) | 2>&1 tee buildtest-with-bmake-and-j-option.log

As you can see, make -j works fine with 2.6.5 but fails with 2.7.0-rc2.

[nobu (Nobuyoshi Nakada)]

Does this log mean the following libraries are stopped by make due to the first failure in cxxanyargs or in yet another branch?

--- ext/-test-/cxxanyargs/all ---
A failure has been detected in another branch of the parallel make

make[2]: stopped in /tmp/ruby-2.7.0-rc2/ext/-test-/cxxanyargs
*** [ext/-test-/cxxanyargs/all] Error code 2

make[1]: stopped in /tmp/ruby-2.7.0-rc2
--- ext/-test-/exception/all ---
A failure has been detected in another branch of the parallel make

make[2]: stopped in /tmp/ruby-2.7.0-rc2/ext/-test-/exception
*** [ext/-test-/exception/all] Error code 2

make[1]: stopped in /tmp/ruby-2.7.0-rc2
--- ext/-test-/enumerator_kw/all ---
A failure has been detected in another branch of the parallel make

make[2]: stopped in /tmp/ruby-2.7.0-rc2/ext/-test-/enumerator_kw
*** [ext/-test-/enumerator_kw/all] Error code 2

make[1]: stopped in /tmp/ruby-2.7.0-rc2
--- ext/psych/all ---
A failure has been detected in another branch of the parallel make

make[2]: stopped in /tmp/ruby-2.7.0-rc2/ext/psych
*** [ext/psych/all] Error code 2

make[1]: stopped in /tmp/ruby-2.7.0-rc2
4 errors

make[1]: stopped in /tmp/ruby-2.7.0-rc2
*** [build-ext] Error code 2

[[e-mail] (Koichiro Iwao)]

• File buildtest-with-bmake-and-j-option.log buildtest-with-bmake-and-j-option.log added
• File buildtest-with-gmake-and-j-option.log buildtest-with-gmake-and-j-option.log added
• Status changed from Closed to Feedback

It is not fixed yet for me, too.

I attach ruby-{2.6.5,2.7.0} build result with {BSD,GNU} make.

[naruse (Yui NARUSE)]

• Status changed from Feedback to Closed

Applied in changeset git|53adb53c9aea5da98ed3e470983b2cbfe367cb7d.

---

Ignore expected errors on compiling C++ source [Bug #16331]

BSD make can run parallel more aggressively than GNU make. It communicate
with other make process through -J option in MAKEFLAGS environment variable
to notify a build failure happend in an other pararell make process.
https://www.freebsd.org/cgi/man.cgi?make

It usually works well but ext/-test-/cxxanyargs/Makefile has two targets
which are expected to fail (failure.o and failurem1.o).

Additional note:
To test and debug this issue, following command will speed up it.
make -f exts.mk -j8 clean all

[naruse (Yui NARUSE)]

• Backport changed from 2.5: REQUIRED, 2.6: REQUIRED to 2.5: REQUIRED, 2.6: REQUIRED, 2.7: REQUIRED

[naruse (Yui NARUSE)]

• Backport changed from 2.5: REQUIRED, 2.6: REQUIRED, 2.7: REQUIRED to 2.5: REQUIRED, 2.6: REQUIRED, 2.7: DONE

ruby_2_7 a930174d798b58ee10493bf192d277ffe08518a7.