Bug #18073
closedtest/ruby/test_jit.rb: "error: invalid use of '__builtin_va_arg_pack ()'" on Ruby 2.7.4 on gcc 4.8.5
Description
I hit the following JIT test failures causing the following error on v2_7_4 tag (Ruby 2.7.4) with gcc 4.8.5 on RHEL 7. The tests work on Ruby 2.7.3 and also works on Ruby 3.0.2.
/mnt/git/ruby/ruby/.ext/include/x86_64-linux/rb_mjit_min_header-2.7.4.h: In function 'sprintf':
/mnt/git/ruby/ruby/.ext/include/x86_64-linux/rb_mjit_min_header-2.7.4.h:442:3: error: invalid use of '__builtin_va_arg_pack ()'
return __builtin___sprintf_chk (__s, 2 - 1,
^
$ which gcc
/usr/bin/gcc
$ gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ git branch
* (detached from v2_7_4)
...
$ autoconf
$ ./configure --enable-shared 2>&1 | tee configure.log
$ make 2>&1 | tee make.log
Here is one of the failures. I would attach the full long files on this ticket.
$ make test-all TESTS="-v test/ruby/test_jit.rb" 2>&1 | tee make_test-all_test_jit.log
...
30) Failure:
TestJIT#test_compile_insn_local [/mnt/git/ruby/ruby/test/ruby/test_jit.rb:62]:
Expected 1 times of JIT success, but succeeded 0 times.
script:
"""
p proc {
foo = 1
foo
}.call
"""
stderr:
"""
/mnt/git/ruby/ruby/.ext/include/x86_64-linux/rb_mjit_min_header-2.7.4.h: In function 'sprintf':
/mnt/git/ruby/ruby/.ext/include/x86_64-linux/rb_mjit_min_header-2.7.4.h:442:3: error: invalid use of '__builtin_va_arg_pack ()'
return __builtin___sprintf_chk (__s, 2 - 1,
^
compilation terminated due to -Wfatal-errors.
Successful MJIT finish
"""
.
<1> expected but was
<0>.
...
Finished tests in 29.837252s, 3.1504 tests/s, 15.7186 assertions/s.
94 tests, 469 assertions, 87 failures, 0 errors, 4 skips
...
Files
Updated by jaruga (Jun Aruga) over 3 years ago
- File configure.log configure.log added
- File make.log make.log added
- File make_test-all_test_jit.log make_test-all_test_jit.log added
Updated by jaruga (Jun Aruga) over 3 years ago
- Subject changed from test/ruby/test_jit.rb: failures "error: invalid use of '__builtin_va_arg_pack ()'" on Ruby 2.7.4 on gcc 4.8.5 on RHEL7 to test/ruby/test_jit.rb: "error: invalid use of '__builtin_va_arg_pack ()'" on Ruby 2.7.4 on gcc 4.8.5
Updated by jaruga (Jun Aruga) over 3 years ago
I confirmed this failures are reproduced at RubyCI - CentOS 7.6(1810) x86_64 - 2.7 case.
http://rubyci.s3.amazonaws.com/centos7/ruby-2.7/log/20210810T041843Z.fail.html.gz
Updated by jaruga (Jun Aruga) over 3 years ago
I executed git bisect
between v2_7_3
and v2_7_4
. And I found the following commit causes these test failures.
Fix 2.7 build
https://github.com/ruby/ruby/commit/29bbad939939c6dceb804aac667ba372fdee4ef5
I checked like this by git bisect
$ cat ~/work/test.sh
#!/bin/bash
set -ex
git clean -fdx
autoconf
./configure --enable-shared --prefix $(pwd)/dest
make
make test-all TESTS="-v test/ruby/test_jit.rb -n TestJIT#test_compile_insn_local"
Then run.
$ git bisect start v2_7_4 v2_7_3
$ git bisect run ~/work/test.sh
...
29bbad939939c6dceb804aac667ba372fdee4ef5 is the first bad commit
...
bisect run success
Updated by jaruga (Jun Aruga) over 3 years ago
- Backport changed from 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN to 2.6: UNKNOWN, 2.7: REQUIRED, 3.0: UNKNOWN
Updated by jaruga (Jun Aruga) over 3 years ago
- Backport changed from 2.6: UNKNOWN, 2.7: REQUIRED, 3.0: UNKNOWN to 2.6: UNKNOWN, 2.7: REQUIRED, 3.0: DONTNEED
Updated by nobu (Nobuyoshi Nakada) over 3 years ago
jaruga (Jun Aruga) wrote in #note-4:
I executed
git bisect
betweenv2_7_3
andv2_7_4
. And I found the following commit causes these test failures.Fix 2.7 build
https://github.com/ruby/ruby/commit/29bbad939939c6dceb804aac667ba372fdee4ef5
This commit is for configure only.
How the results of configure
(config.status and config.h files) differ before and after this commit?
Updated by jaruga (Jun Aruga) over 3 years ago
This commit is for configure only.
How the results of configure (config.status and config.h files) differ before and after this commit?
I did ssh login to RubyCI CentOS7 server, and checked the files between commits fd95a18059
(previous commit for 29bbad9399
) and 29bbad9399
(first failure),
config.status
./.ext/include/x86_64-linux/ruby/config.h
by running the following commands.
git clean -fdx
autoconf
./configure --enable-shared
For the config.status
, here is the result of the difference. There are some differences by not sorted items.
$ \diff -u ok.fd95a18059.config.status err.29bbad9399.config.status
--- ok.fd95a18059.config.status 2021-08-10 10:38:05.000139654 +0000
+++ err.29bbad9399.config.status 2021-08-10 10:25:19.967182125 +0000
@@ -791,18 +791,18 @@
S["GNU_LD"]="yes"
S["LD"]="ld"
S["GCC"]="yes"
-S["EGREP"]="/bin/grep -E"
-S["GREP"]="/bin/grep"
-S["CPP"]="$(CC) -E"
S["ac_ct_CXX"]=""
S["CXXFLAGS"]=""
S["CXX"]="g++"
+S["EGREP"]="/bin/grep -E"
+S["GREP"]="/bin/grep"
+S["CPP"]="gcc -E"
S["OBJEXT"]="o"
S["EXEEXT"]=""
+S["ac_ct_CC"]="gcc"
S["CPPFLAGS"]=" $(DEFS) ${cppflags}"
S["LDFLAGS"]="-L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic"
-S["CFLAGS"]="${cflags} -fPIC"
-S["ac_ct_CC"]="gcc"
+S["CFLAGS"]="-g -O2 -fPIC"
S["CC"]="gcc -std=gnu99"
S["target_os"]="linux"
S["target_vendor"]="pc"
However the essential difference is the following part.
-S["CFLAGS"]="${cflags} -fPIC"
+S["CFLAGS"]="-g -O2 -fPIC"
For the config.h
, here is the difference.
$ \diff -u ok.fd95a18059.config.h err.29bbad9399.config.h
--- ok.fd95a18059.config.h 2021-08-10 10:38:04.799141259 +0000
+++ err.29bbad9399.config.h 2021-08-10 10:25:19.775183599 +0000
@@ -169,11 +169,8 @@
#define PRI_SIZE_PREFIX "z"
#define PRI_PTRDIFF_PREFIX "t"
#define HAVE_STRUCT_STAT_ST_BLKSIZE 1
-#define HAVE_ST_BLKSIZE 1
#define HAVE_STRUCT_STAT_ST_BLOCKS 1
-#define HAVE_ST_BLOCKS 1
#define HAVE_STRUCT_STAT_ST_RDEV 1
-#define HAVE_ST_RDEV 1
#define SIZEOF_STRUCT_STAT_ST_SIZE SIZEOF_OFF_T
#define SIZEOF_STRUCT_STAT_ST_BLOCKS SIZEOF_OFF_T
#define SIZEOF_STRUCT_STAT_ST_INO SIZEOF_LONG
Updated by nobu (Nobuyoshi Nakada) over 3 years ago
jaruga (Jun Aruga) wrote in #note-8:
However the essential difference is the following part.
-S["CFLAGS"]="${cflags} -fPIC" +S["CFLAGS"]="-g -O2 -fPIC"
${cflags}
should contain ${optflags} ${debugflags} ${warnflags}
by the default.
That means the latter CFLAGS
lacks these flags.
Possibly it might affect the precompiled header.
Updated by jaruga (Jun Aruga) over 3 years ago
${cflags}
should contain ${optflags} ${debugflags} ${warnflags} by the default.
That means the latter CFLAGS lacks these flags.
Possibly it might affect the precompiled header.
@nobu (Nobuyoshi Nakada) OK. Could you provide a patch to change only from S["CFLAGS"]="-g -O2 -fPIC"
to S["CFLAGS"]="${cflags} -fPIC"
in config.status
? Then I am happy to test the patch or you can test it on the RubyCI CentOS7 server. Thanks.
Updated by nobu (Nobuyoshi Nakada) over 3 years ago
jaruga (Jun Aruga) wrote in #note-10:
@nobu (Nobuyoshi Nakada) OK. Could you provide a patch to change only from
S["CFLAGS"]="-g -O2 -fPIC"
toS["CFLAGS"]="${cflags} -fPIC"
inconfig.status
? Then I am happy to test the patch or you can test it on the RubyCI CentOS7 server. Thanks.
It should be the default, unless you set CFLAGS
at the command line or as the environment variable.
Updated by jaruga (Jun Aruga) over 3 years ago
It should be the default, unless you set CFLAGS at the command line or as the environment variable.
As we chat, I confirmed the environment variable CFLAGS
is not set before running autoconf
, ./configure
again. The CFLAGS: ${optflags} ${debugflags} ${warnflags}
is printed as an output of the ./configure
, but the actual value is S["CFLAGS"]="-g -O2 -fPIC"
in config.status
.
$ git checkout 29bbad939939c6dceb804aac667ba372fdee4ef5
$ git clean -fdx
$ echo "CFLAGS: $CFLAGS"
CFLAGS:
$ env | grep CFLAGS
$ echo $?
1
$ unset CFLAGS
$ echo $?
0
$ autoconf
$ ./configure --enable-shared 2>&1 | tee configure.log
...
---
Configuration summary for ruby version 2.7.4
...
* CFLAGS: ${optflags} ${debugflags} ${warnflags}
...
$ grep '"CFLAGS"' config.status
S["CFLAGS"]="-g -O2 -fPIC"
I also compared the difference of the outputs of the ./configure --enable-shared
, sorting the result by the sort
command to only see the essential differences. And here is the result of the difference.
$ \diff -u ok.fd95a18059.configure_sorted.log err.29bbad9399.configure_sorted.log
--- ok.fd95a18059.configure_sorted.log 2021-08-11 09:10:44.858148283 +0000
+++ err.29bbad9399.configure_sorted.log 2021-08-11 09:11:11.689938763 +0000
@@ -507,7 +507,7 @@
checking grp.h presence... yes
checking grp.h usability... yes
checking host system type... x86_64-pc-linux-gnu
-checking how to run the C preprocessor... gcc -std=gnu99 -E
+checking how to run the C preprocessor... gcc -E
checking ieeefp.h presence... no
checking ieeefp.h usability... no
checking if enum over int is allowed... yes
@@ -543,7 +543,6 @@
checking pthread.h usability... yes
checking pwd.h presence... yes
checking pwd.h usability... yes
-checking return type of signal handlers... void
checking sanitizer/asan_interface.h presence... no
checking sanitizer/asan_interface.h usability... no
checking sanitizer/msan_interface.h presence... no
@@ -694,7 +693,7 @@
checking whether dtrace USDT is available... no
checking whether finite is declared... yes
checking whether g++ accepts -g... no
-checking whether gcc -std=gnu99 -E accepts -o... yes
+checking whether gcc -E accepts -o... yes
checking whether gcc -std=gnu99 needs -traditional... no
checking whether gcc accepts -g... yes
checking whether getenv is declared... yes
Updated by xtkoba (Tee KOBAYASHI) over 3 years ago
It seems that in configure.ac
, AC_PROG_CC_C99
must come before AC_PREPROC_IFELSE
to set CPP
correctly. That is,
AC_INIT
ac_cv_prog_cc_c99=-std=gnu99
AC_PROG_CC_C99
AC_PREPROC_IFELSE([AC_LANG_SOURCE([[]])],[],[])
echo "CC:" "$CC"
echo "CPP:" "$CPP"
works fine, while
AC_INIT
ac_cv_prog_cc_c99=-std=gnu99
AC_PREPROC_IFELSE([AC_LANG_SOURCE([[]])],[],[])
AC_PROG_CC_C99
echo "CC:" "$CC"
echo "CPP:" "$CPP"
does not.
The relevant part of configure.ac
diff is shown below:
--- ruby-2.7.3/configure.ac
+++ ruby-2.7.4/configure.ac
@@ -177,12 +180,14 @@
# clang version 1.0 (http://llvm.org/svn/llvm-project/cfe/tags/Apple/clang-23 exported)
# Apple clang version 2.0 (tags/Apple/clang-137) (based on LLVM 2.9svn)
# Apple clang version 2.1 (tags/Apple/clang-163.7.1) (based on LLVM 3.0svn)
- AS_IF([! $CC -E -xc - <<SRC >/dev/null], [
- @%:@if defined __APPLE_CC__ && defined __clang_major__ && __clang_major__ < 3
- @%:@error premature clang
- @%:@endif
-SRC
- AC_MSG_ERROR([clang version 3.0 or later is required])
+ AC_PREPROC_IFELSE(
+ [AC_LANG_PROGRAM([
+ @%:@if defined __APPLE_CC__ && defined __clang_major__ && __clang_major__ < 3
+ @%:@error premature clang
+ @%:@endif
+ ])],
+ [],
+ [AC_MSG_ERROR([clang version 3.0 or later is required])
])],
[openbsd*:openbsd*], [
AC_CHECK_TOOLS(CC, [cc])
@@ -191,7 +196,9 @@
AC_CHECK_TOOL(CC, gcc)
])
-AC_PROG_CC_C99
+dnl Seems necessarily in order to add -std=gnu99 option for gcc 4.9.
+m4_version_prereq([2.70], [], [AC_PROG_CC_C99])
+
AS_CASE([$CC],
[gcc-*], [
gcc_prefix=gcc- gcc_suffix=`echo "$CC" | sed 's/^gcc//'`
Updated by jaruga (Jun Aruga) over 3 years ago
@xtkoba (Tee KOBAYASHI) thanks for the info. I confirmed the value of CPP
on RHEL 7 autoconf version 2.69.
The following configure.ac
works fine.
$ cat configure.ac
AC_INIT
ac_cv_prog_cc_c99=-std=gnu99
AC_PROG_CC_C99
AC_PREPROC_IFELSE([AC_LANG_SOURCE([[]])],[],[])
echo "CC:" "$CC"
echo "CPP:" "$CPP"
$ autoconf
$ ./configure
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for gcc option to accept ISO C99... (cached) -std=gnu99
checking how to run the C preprocessor... gcc -std=gnu99 -E
CC: gcc -std=gnu99
CPP: gcc -std=gnu99 -E
The following configure.ac
doesn't.
$ cat configure.ac
AC_INIT
ac_cv_prog_cc_c99=-std=gnu99
AC_PREPROC_IFELSE([AC_LANG_SOURCE([[]])],[],[])
AC_PROG_CC_C99
echo "CC:" "$CC"
echo "CPP:" "$CPP"
$ autoconf
$ ./configure
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E
checking for gcc option to accept ISO C99... (cached) -std=gnu99
CC: gcc -std=gnu99
CPP: gcc -E
Updated by jaruga (Jun Aruga) over 3 years ago
jaruga (Jun Aruga) wrote in #note-4:
I executed
git bisect
betweenv2_7_3
andv2_7_4
. And I found the following commit causes these test failures.Fix 2.7 build
https://github.com/ruby/ruby/commit/29bbad939939c6dceb804aac667ba372fdee4ef5
As a temporary workaround to build Ruby 2.7.4 on RHEL7, I am applying a patch to revert above commit, and I see the test failures in test/ruby/test_jit.rb are gone.
Updated by ivoanjo (Ivo Anjo) almost 3 years ago
I run into a similar issue (in https://github.com/DataDog/dd-trace-rb/issues/1799); in my case, I was running the Semaphore CI Ruby image for 2.7.5 and when trying to run with JIT I get the same failures
semaphore@semaphore-vm:~$ ruby -v
ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]
semaphore@semaphore-vm:~$ gcc -v 2>&1 | grep "gcc version"
gcc version 4.8.5 (Ubuntu 4.8.5-4ubuntu8)
semaphore@semaphore-vm:~$ ruby --jit --jit-warnings --jit-debug --jit-verbose=2 -e 'def foo; end; while true; foo; end'
MJIT: CC defaults to /usr/bin/gcc
MJIT: tmp_dir is /tmp
Creating precompiled header
Starting process: /usr/bin/gcc /usr/bin/gcc -std=gnu99 -w -Wfatal-errors -fPIC -shared -w -pipe -ggdb3 -nodefaultlibs -nostdlib -o /tmp/_ruby_mjit_hp4368u0.h.gch /home/semaphore/.rbenv/versions/2.7.5/include/ruby-2.7.0/x86_64-linux/rb_mjit_min_header-2.7.5.h
start compilation: foo@-e:1 -> /tmp/_ruby_mjit_p4368u1.c
Starting process: /usr/bin/gcc /usr/bin/gcc -std=gnu99 -w -Wfatal-errors -fPIC -shared -w -pipe -ggdb3 -o /tmp/_ruby_mjit_p4368u1.o /tmp/_ruby_mjit_p4368u1.c -c -nostartfiles -nodefaultlibs -nostdlib
/home/semaphore/.rbenv/versions/2.7.5/include/ruby-2.7.0/x86_64-linux/rb_mjit_min_header-2.7.5.h: In function 'sprintf':
/home/semaphore/.rbenv/versions/2.7.5/include/ruby-2.7.0/x86_64-linux/rb_mjit_min_header-2.7.5.h:406:3: error: invalid use of '__builtin_va_arg_pack ()'
return __builtin___sprintf_chk (__s, 2 - 1,
^
compilation terminated due to -Wfatal-errors.
compile_c_to_o: compile error: 1
Failed to generate so: /tmp/_ruby_mjit_p4368u1.so
start compilation: block in already_loaded?@/home/semaphore/.rbenv/versions/2.7.5/lib/ruby/site_ruby/2.7.0/rubygems.rb:1290 -> /tmp/_ruby_mjit_p4368u0.c
Starting process: /usr/bin/gcc /usr/bin/gcc -std=gnu99 -w -Wfatal-errors -fPIC -shared -w -pipe -ggdb3 -o /tmp/_ruby_mjit_p4368u0.o /tmp/_ruby_mjit_p4368u0.c -c -nostartfiles -nodefaultlibs -nostdlib
/home/semaphore/.rbenv/versions/2.7.5/include/ruby-2.7.0/x86_64-linux/rb_mjit_min_header-2.7.5.h: In function 'sprintf':
/home/semaphore/.rbenv/versions/2.7.5/include/ruby-2.7.0/x86_64-linux/rb_mjit_min_header-2.7.5.h:406:3: error: invalid use of '__builtin_va_arg_pack ()'
return __builtin___sprintf_chk (__s, 2 - 1,
^
compilation terminated due to -Wfatal-errors.
compile_c_to_o: compile error: 1
Failed to generate so: /tmp/_ruby_mjit_p4368u0.so
Would it make sense to perhaps validate that the mjit header is compilable after generating it?
Updated by jeremyevans0 (Jeremy Evans) about 1 year ago
- Status changed from Open to Closed
These failures look related to MJIT, which has been removed.