Backport #7959: configure contains non-portable shell code

• Status: Rejected
• Assignee: nagachika (Tomoyuki Chikanaga)

[jeremyevans0 (Jeremy Evans)]

On 2.0.0 and trunk, configure contains non-portable shell code. When run on OpenBSD, configure emits the following errors and results in a ruby that builds but cannot run without manually specifying -I for the ruby lib directories:

...
checking for prefix of external symbols... NONE
checking for pthread.h... (cached) yes
checking if make is GNU make... no
checking for safe null command for make... :
checking for memmem... no
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g;s/ ...: invalid command code "
././configure: s/\//g: not found
././configure: s/^//;s/$//": not found
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g;s/ ...: invalid command code "
././configure: s/\//g: not found
././configure: s/^//;s/$//": not found
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g;s/ ...: invalid command code "
././configure: s/\//g: not found
././configure: s/^//;s/$//": not found
sed: 1: ""s/${[^${}"]}/"&"/g"": invalid command code "
sed: 1: ""s/${[^${}"]}/"&"/g;s/ ...: invalid command code "
././configure: s/\//g: not found
././configure: s/^//;s/$//": not found
checking for nroff... no
checking for awf... no
.ext/include/x86_64-openbsd/ruby/config.h updated
verconf.h updated
ruby library version = 2.0
configure: creating ./config.status
config.status: creating Makefile
config.status: creating ruby-2.0.pc

The last revision that worked correctly was 39288.

It's trivial to work around the issue by using "bash configure" instead of "./configure", but that shouldn't be necessary. I'm sorry for not reporting this before the official release of 2.0.0, but this issue was introduced after rc2.

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed
• % Done changed from 0 to 100

This issue was solved with changeset r39506.
Jeremy, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

---

configure.in: non-portable shell behavior

• configure.in (unexpand_shvar): get rid of non-portable shell
  behavior on OpenBSD, so no extra quotes. [Bug #7959]

[nobu (Nobuyoshi Nakada)]

• Tracker changed from Bug to Backport
• Project changed from Ruby master to Backport200
• Category deleted (build)
• Status changed from Closed to Assigned
• Assignee changed from nobu (Nobuyoshi Nakada) to nagachika (Tomoyuki Chikanaga)

[nagachika (Tomoyuki Chikanaga)]

nobu san

Should the following line in configure.in be eliminated?

l.3287 quote=['']

[nobu (Nobuyoshi Nakada)]

Yes, it should not be there anymore, I missed to remove it.

[nagachika (Tomoyuki Chikanaga)]

• Status changed from Assigned to Closed

This issue was solved with changeset r39709.
Jeremy, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

---

merge revision(s) 39506,39514: [Backport #7959]

* configure.in (unexpand_shvar): get rid of non-portable shell
  behavior on OpenBSD, so no extra quotes.  [Bug #7959]

[kernigh (George Koehler)]

=begin
Hello, Rubyists. This fix is not working for me. I am still getting this bug, or a very similar bug.

I am running OpenBSD 5.2 and autoconf 2.69. I follow trunk and ruby_2_0_0 with git. Here are my ruby -v

• ruby 2.1.0dev (2013-03-13 trunk 39737) [x86_64-openbsd5.2]
• ruby 2.0.0p57 (2013-03-12 revision 39728) [x86_64-openbsd5.2]

When I configure trunk, I see errors about "no closing quote".

$ ../configure --prefix=$HOME/prefix
--with-baseruby=/usr/local/bin/ruby19
CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib
...
checking if make is GNU make... no
checking for safe null command for make... :
checking for memmem... no
../configure: no closing quote
../configure: no closing quote
../configure: no closing quote
../configure: no closing quote
checking for nroff... no
checking for awf... no
.ext/include/x86_64-openbsd5.2/ruby/config.h unchanged
verconf.h unchanged
ruby library version = 2.1.0
configure: creating ./config.status
config.status: creating Makefile
config.status: creating ruby-2.1.pc

Then verconf.h looks like a mess:

#define RUBY_BASE_NAME "ruby"
#define RUBY_VERSION_NAME RUBY_BASE_NAME"-"RUBY_LIB_VERSION
#define RUBY_LIB_VERSION_STYLE 3 /* full */
#define RUBY_EXEC_PREFIX "/home/kernigh/prefix"
#define RUBY_LIB_PREFIX "arch"RUBY_EXEC_PREFIX"/lib/"RUBY_BASE_NAME"arch"
#define RUBY_ARCH_PREFIX_FOR(arch) "arch"RUBY_LIB_PREFIX"/"arch"arch"
#define RUBY_SITEARCH_PREFIX_FOR(arch) "arch"RUBY_LIB_PREFIX"/"arch"arch"
#define RUBY_LIB "arch"RUBY_LIB_PREFIX"/"RUBY_LIB_VERSION"arch"
#define RUBY_ARCH_LIB_FOR(arch) "arch"RUBY_LIB"/"arch"arch"
#define RUBY_SITE_LIB "arch"RUBY_LIB_PREFIX"/site_ruby"
#define RUBY_SITE_ARCH_LIB_FOR(arch) "arch"RUBY_SITE_LIB2"/"arch"arch"
#define RUBY_VENDOR_LIB "arch"RUBY_LIB_PREFIX"/vendor_ruby"
#define RUBY_VENDOR_ARCH_LIB_FOR(arch) "arch"RUBY_VENDOR_LIB2"/"arch"arch"
#define RUBY_PLATFORM "x86_64-openbsd5.2"

This ruby can't find the installed libraries:

$ ./ruby -v
ruby 2.1.0dev (2013-03-12 trunk 39730) [x86_64-openbsd5.2]
$ ./ruby -e 'p $:'
internal:gem_prelude:1:in require': cannot load such file -- rubygems.rb (LoadError) from <internal:gem_prelude>:1:in '
$ ./ruby --disable=gems -e 'p $:'
["archarch/home/kernigh/prefix/lib/rubyarch/site_ruby/2.1.0", "archarcharch/home/kernigh/prefix/lib/rubyarch/site_ruby/2.1.0/x86_64-openbsd5.2arch", "archarch/home/kernigh/prefix/lib/rubyarch/site_ruby", "archarch/home/kernigh/prefix/lib/rubyarch/vendor_ruby/2.1.0", "archarcharch/home/kernigh/prefix/lib/rubyarch/vendor_ruby/2.1.0/x86_64-openbsd5.2arch", "archarch/home/kernigh/prefix/lib/rubyarch/vendor_ruby", "arch/home/kernigh/prefix/lib/rubyarch/2.1.0", "archarch/home/kernigh/prefix/lib/rubyarch/2.1.0/x86_64-openbsd5.2arch"]

When I configure ruby_2_0_0, the problem is slightly different:

$ ../configure --prefix=$HOME/prefix
--with-baseruby=/usr/local/bin/ruby19
CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib
...
checking if make is GNU make... no
checking for safe null command for make... :
checking for memmem... no
sed: 1: ""s/${[^${}"]}/"&"/g;;s ...: invalid command code "
../configure: s/\//g: not found
../configure: s/^//;s/$//": not found
sed: 1: ""s/${[^${}"]}/"&"/g;;s ...: invalid command code "
../configure: s/\//g: not found
../configure: s/^//;s/$//": not found
sed: 1: ""s/${[^${}"]}/"&"/g;;s ...: invalid command code "
../configure: s/\//g: not found
../configure: s/^//;s/$//": not found
sed: 1: ""s/${[^${}"]}/"&"/g;;s ...: invalid command code "
../configure: s/\//g: not found
../configure: s/^//;s/$//": not found
checking for nroff... no
checking for awf... no
.ext/include/x86_64-openbsd5.2/ruby/config.h unchanged
verconf.h unchanged
ruby library version = 2.0.0
configure: creating ./config.status
config.status: creating Makefile
config.status: creating ruby-2.0.pc

#define RUBY_BASE_NAME "ruby"
#define RUBY_VERSION_NAME RUBY_BASE_NAME"-"RUBY_LIB_VERSION
#define RUBY_LIB_VERSION_STYLE 3 /* full */
#define RUBY_LIB_PREFIX "arch"
#define RUBY_ARCH_PREFIX_FOR(arch) "arch"
#define RUBY_SITEARCH_PREFIX_FOR(arch) "arch"
#define RUBY_LIB "arch"RUBY_LIB_PREFIX"/"RUBY_LIB_VERSION"arch"
#define RUBY_ARCH_LIB_FOR(arch) "arch"RUBY_LIB"/"arch"arch"
#define RUBY_SITE_LIB "arch"RUBY_LIB_PREFIX"/site_ruby"
#define RUBY_SITE_ARCH_LIB_FOR(arch) "arch"RUBY_SITE_LIB2"/"arch"arch"
#define RUBY_VENDOR_LIB "arch"RUBY_LIB_PREFIX"/vendor_ruby"
#define RUBY_VENDOR_ARCH_LIB_FOR(arch) "arch"RUBY_VENDOR_LIB2"/"arch"arch"
#define RUBY_PLATFORM "x86_64-openbsd5.2"

$ ./ruby -v
ruby 2.0.0p57 (2013-03-12 revision 39728) [x86_64-openbsd5.2]
$ ./ruby -e 'p $:'
internal:gem_prelude:1:in require': cannot load such file -- rubygems.rb (LoadError) from <internal:gem_prelude>:1:in '
$ ./ruby --disable=gems -e 'p $:'
["archarch/site_ruby/2.0.0", "archarcharch/site_ruby/2.0.0/x86_64-openbsd5.2arch", "archarch/site_ruby", "archarch/vendor_ruby/2.0.0", "archarcharch/vendor_ruby/2.0.0/x86_64-openbsd5.2arch", "archarch/vendor_ruby", "arch/2.0.0", "archarch/2.0.0/x86_64-openbsd5.2arch"]

Like jeremyevans0, I can work around this issue by using bash. I run "bash ../configure ..." and rebuild trunk Ruby, then trunk Ruby no longer has this problem.

I don't know where in configure causes this problem. I am guessing that the problem might be in shvar_to_cpp, but I am not sure. OpenBSD /bin/sh is a modified pdksh. To try to reproduce this problem with another system, one might try using pdksh. Also, bug #8071 reported a similar problem with ksh93, so one might try using ksh93.

=end

[jeremyevans0 (Jeremy Evans)]

It appears this bug is still present in ruby 2.0.0-p195 and trunk. Bisecting v2_0_0_rc2 and trunk, the last revision that worked correctly is still 39288. Can this bug please be reopened? I'll be happy to test patches.

ruby 2.0.0-p195 on OpenBSD doesn't even build if you configure with /bin/sh and not with bash, due to issues with RUBY_EXEC_PREFIX:

cc -O2 -pipe -fPIC -include ruby/config.h -include ruby/missing.h -D_FORTIFY_SOURCE=2 -fstack-protector -fno-strict-overflow -fvisibility=hidden -DRUBY_EXPORT -DOPENSSL_NO_STATIC_ENGINE -I/usr/local/include -I. -I.ext/include/i386-openbsd -I./include -I. -o ruby.o -c ruby.c
ruby.c: In function 'ruby_init_loadpath_safe':
ruby.c:495: error: expected expression before ';' token
*** Error 1 in /usr/obj/ports/ruby-2.0.0-p195/ruby-2.0.0-p195 (Makefile:326 'ruby.o')

[luislavena (Luis Lavena)]

• Status changed from Closed to Assigned
• Assignee changed from nagachika (Tomoyuki Chikanaga) to nobu (Nobuyoshi Nakada)

[luislavena (Luis Lavena)]

• % Done changed from 100 to 50

[naruse (Yui NARUSE)]

• Tracker changed from Backport to Bug
• Project changed from Backport200 to Ruby master

[nobu (Nobuyoshi Nakada)]

• Status changed from Assigned to Closed
• % Done changed from 50 to 100

This issue was solved with changeset r40758.
Jeremy, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

---

configure.in: shvar_to_cpp in ruby

• configure.in: save configured load path values into verconf.in.
• common.mk (verconf.h): create from verconf.in with shvar_to_cpp.rb.
• tool/shvar_to_cpp.rb: turn shell variables into C macros.
  [Bug #7959]

[jeremyevans0 (Jeremy Evans)]

I tested trunk and ./configure appears to work correctly on OpenBSD (no more error messages). Can these changes please be backported to 2.0.0?

[nagachika (Tomoyuki Chikanaga)]

• Tracker changed from Bug to Backport
• Project changed from Ruby master to Backport200
• Status changed from Closed to Assigned
• Assignee changed from nobu (Nobuyoshi Nakada) to nagachika (Tomoyuki Chikanaga)

[mpapis (Michal Papis)]

still getting errors on ruby_2_0_0, any progress on fixing it?

[nagachika (Tomoyuki Chikanaga)]

Backporting r40758 to ruby_2_0_0 cause severe conflict. It seems depend on several changesets in trunk include splitting loadpath.c and strategy of creating verconf.h.

nobu, could you make a backport patch? Or is there any other approach to fix this issue on 2.0.0?

[nobu (Nobuyoshi Nakada)]

r39171,r39296,r39333,r39334,r39345,r39347,r39348,r40261,r40756,r40757,r40758,r40762,r40763,r40770,r40771,r40777,r40779,r40784,r41352,r41357

[mpapis (Michal Papis)]

=begin
any chance to get this merged somehow? I get merge conflicts trying to apply some of the revisions on 2.0.0-p247 source
=end

[naruse (Yui NARUSE)]

• Status changed from Assigned to Rejected