Bug #19233
closedFailed to install sqlite3 gem since 7f1ca666424849134990d022266bcd4d6636465f using Docker
Description
Failed to install sqlite3 gem since 7f1ca666424849134990d022266bcd4d6636465f using Docker. This issue comes from https://github.com/rails/rails/pull/46711 .
Steps to reproduce¶
- Install Docker
- Follow this step
git clone https://github.com/yahonda/rep19189
cd rep19189
docker build .
Expected behavior¶
It should pass.
Actual behavior¶
It fails Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
$ docker build .
Sending build context to Docker daemon 69.63kB
Step 1/5 : FROM rubylang/rubyfarm:7f1ca666424849134990d022266bcd4d6636465f
---> cdd68058177d
Step 2/5 : RUN apt-get update -qq && apt-get install -y sqlite3 libsqlite3-dev pkg-config
---> Using cache
---> b8928c4f6425
Step 3/5 : WORKDIR /myapp
---> Using cache
---> bae259f8abf3
Step 4/5 : COPY Gemfile /myapp/Gemfile
---> Using cache
---> a9fe0822ad64
Step 5/5 : RUN bundle install
---> Running in 00cd9fc0e691
Don't run Bundler as root. Installing your bundle as root will break this
application for all non-root users on this machine.
Fetching gem metadata from https://rubygems.org/.....
Resolving dependencies...
Using bundler 2.4.0.dev
Fetching mini_portile2 2.8.0
Installing mini_portile2 2.8.0
Fetching sqlite3 1.5.4
Installing sqlite3 1.5.4 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory:
/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ext/sqlite3
/opt/ruby/bin/ruby -I /opt/ruby/lib/ruby/3.2.0+2 extconf.rb
Ignoring debug-1.6.2 because its extensions are not built. Try: gem pristine
debug --version 1.6.2
Building sqlite3-ruby using packaged sqlite3.
Extracting sqlite-autoconf-3400000.tar.gz into
tmp/x86_64-linux-gnu/ports/sqlite3/3.40.0... OK
Running 'configure' for sqlite3 3.40.0... OK
Running 'compile' for sqlite3 3.40.0... OK
Running 'install' for sqlite3 3.40.0... OK
Activating sqlite3 3.40.0 (from
/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0)...
Could not configure the build properly (pkg_config). Please install either the
`pkg-config` utility or the `pkg-config` rubygem.
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/opt/ruby/bin/$(RUBY_BASE_NAME)
--help
--download-dependencies
--with-sqlcipher
--without-sqlcipher
--with-sqlcipher-dir
--without-sqlcipher-dir
--with-sqlcipher-include
--without-sqlcipher-include
--with-sqlcipher-lib
--without-sqlcipher-lib
--enable-system-libraries
--disable-system-libraries
--with-sqlcipher
--without-sqlcipher
--with-sqlcipher-dir
--without-sqlcipher-dir
--with-sqlcipher-include
--without-sqlcipher-include
--with-sqlcipher-lib
--without-sqlcipher-lib
--with-sqlite-source-dir
--with-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-dir
--without-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-dir
--with-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-include
--without-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-include=${/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-dir}/include
--with-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-lib
--without-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-lib=${/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-dir}/lib
--with-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-config
--without-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-config
--with-pkg-config
--without-pkg-config
To see why this extension failed to compile, please check the mkmf.log which can
be found here:
/opt/ruby/lib/ruby/gems/3.2.0+2/extensions/x86_64-linux/3.2.0+2/sqlite3-1.5.4/mkmf.log
extconf failed, exit code 1
Gem files will remain installed in
/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4 for inspection.
Results logged to
/opt/ruby/lib/ruby/gems/3.2.0+2/extensions/x86_64-linux/3.2.0+2/sqlite3-1.5.4/gem_make.out
/opt/ruby/lib/ruby/3.2.0+2/rubygems/ext/builder.rb:102:in `run'
/opt/ruby/lib/ruby/3.2.0+2/rubygems/ext/ext_conf_builder.rb:28:in `build'
/opt/ruby/lib/ruby/3.2.0+2/rubygems/ext/builder.rb:171:in `build_extension'
/opt/ruby/lib/ruby/3.2.0+2/rubygems/ext/builder.rb:205:in `block in
build_extensions'
/opt/ruby/lib/ruby/3.2.0+2/rubygems/ext/builder.rb:202:in `each'
/opt/ruby/lib/ruby/3.2.0+2/rubygems/ext/builder.rb:202:in `build_extensions'
/opt/ruby/lib/ruby/3.2.0+2/rubygems/installer.rb:843:in `build_extensions'
/opt/ruby/lib/ruby/3.2.0+2/bundler/rubygems_gem_installer.rb:72:in
`build_extensions'
/opt/ruby/lib/ruby/3.2.0+2/bundler/rubygems_gem_installer.rb:28:in `install'
/opt/ruby/lib/ruby/3.2.0+2/bundler/source/rubygems.rb:202:in `install'
/opt/ruby/lib/ruby/3.2.0+2/bundler/installer/gem_installer.rb:54:in `install'
/opt/ruby/lib/ruby/3.2.0+2/bundler/installer/gem_installer.rb:16:in
`install_from_spec'
/opt/ruby/lib/ruby/3.2.0+2/bundler/installer/parallel_installer.rb:186:in
`do_install'
/opt/ruby/lib/ruby/3.2.0+2/bundler/installer/parallel_installer.rb:177:in
`block in worker_pool'
/opt/ruby/lib/ruby/3.2.0+2/bundler/worker.rb:62:in `apply_func'
/opt/ruby/lib/ruby/3.2.0+2/bundler/worker.rb:57:in `block in process_queue'
/opt/ruby/lib/ruby/3.2.0+2/bundler/worker.rb:54:in `loop'
/opt/ruby/lib/ruby/3.2.0+2/bundler/worker.rb:54:in `process_queue'
/opt/ruby/lib/ruby/3.2.0+2/bundler/worker.rb:91:in `block (2 levels) in
create_threads'
An error occurred while installing sqlite3 (1.5.4), and Bundler cannot continue.
In Gemfile:
sqlite3
The command '/bin/sh -c bundle install' returned a non-zero code: 5
Updated by hsbt (Hiroshi SHIBATA) almost 2 years ago
- Related to Bug #19189: Ruby 3.1.3/3.2.x can no longer find pkg-config if not present at buildtime added
Updated by yahonda (Yasuo Honda) almost 2 years ago
There are other findings so far:
- It works fine with Ruby installed locally without Docker.
- It also works fine using
gem "sqlite3", "< 1.5"
Updated by S_H_ (Shun Hiraoka) almost 2 years ago
Hi.
As a test, I also built a container as follows
FROM rubylang/rubyfarm:7f1ca666424849134990d022266bcd4d6636465f
# FROM rubylang/rubyfarm:88c12a29372e3f908190c6af0aa1b2ac6b78fbd1 # passes
# FROM rubylang/rubyfarm:7f1ca666424849134990d022266bcd4d6636465f # fails
# FROM rubylang/ruby:master-nightly-focal # fails
RUN apt-get update -qq && apt-get install -y sqlite3 libsqlite3-dev pkg-config
WORKDIR /myapp
COPY Gemfile /myapp/Gemfile
Then I ran bundle install in the container and got the same error.
sh@DESKTOP-L0NI312:~/rubydev/rep19189$ docker build . -t rep19189:without-bundle-install
...
sh@DESKTOP-L0NI312:~/rubydev/rep19189$ docker run -it rep19189:without-bundle-install
root@0f6123678405:/myapp# bundle install
Don't run Bundler as root. Installing your bundle as root will break this application for all non-root users on this machine.
Fetching gem metadata from https://rubygems.org/.....
Resolving dependencies...
Fetching mini_portile2 2.8.0
Using bundler 2.4.0.dev
Installing mini_portile2 2.8.0
Fetching sqlite3 1.5.4
Installing sqlite3 1.5.4 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ext/sqlite3
/opt/ruby/bin/ruby -I /opt/ruby/lib/ruby/3.2.0+2 extconf.rb
Ignoring debug-1.6.2 because its extensions are not built. Try: gem pristine debug --version 1.6.2
Building sqlite3-ruby using packaged sqlite3.
Extracting sqlite-autoconf-3400000.tar.gz into tmp/x86_64-linux-gnu/ports/sqlite3/3.40.0... OK
Running 'configure' for sqlite3 3.40.0... OK
Running 'compile' for sqlite3 3.40.0... OK
Running 'install' for sqlite3 3.40.0... OK
Activating sqlite3 3.40.0 (from /opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0)...
Could not configure the build properly (pkg_config). Please install either the `pkg-config` utility or the `pkg-config` rubygem.
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/opt/ruby/bin/$(RUBY_BASE_NAME)
--help
--download-dependencies
--with-sqlcipher
--without-sqlcipher
--with-sqlcipher-dir
--without-sqlcipher-dir
--with-sqlcipher-include
--without-sqlcipher-include
--with-sqlcipher-lib
--without-sqlcipher-lib
--enable-system-libraries
--disable-system-libraries
--with-sqlcipher
--without-sqlcipher
--with-sqlcipher-dir
--without-sqlcipher-dir
--with-sqlcipher-include
--without-sqlcipher-include
--with-sqlcipher-lib
--without-sqlcipher-lib
--with-sqlite-source-dir
--with-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-dir
--without-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-dir
--with-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-include
--without-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-include=${/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-dir}/include
--with-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-lib
--without-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-lib=${/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-dir}/lib
--with-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-config
--without-/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc-config
--with-pkg-config
--without-pkg-config
To see why this extension failed to compile, please check the mkmf.log which can be found here:
/opt/ruby/lib/ruby/gems/3.2.0+2/extensions/x86_64-linux/3.2.0+2/sqlite3-1.5.4/mkmf.log
extconf failed, exit code 1
Gem files will remain installed in /opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4 for inspection.
Results logged to /opt/ruby/lib/ruby/gems/3.2.0+2/extensions/x86_64-linux/3.2.0+2/sqlite3-1.5.4/gem_make.out
/opt/ruby/lib/ruby/3.2.0+2/rubygems/ext/builder.rb:102:in `run'
/opt/ruby/lib/ruby/3.2.0+2/rubygems/ext/ext_conf_builder.rb:28:in `build'
/opt/ruby/lib/ruby/3.2.0+2/rubygems/ext/builder.rb:171:in `build_extension'
/opt/ruby/lib/ruby/3.2.0+2/rubygems/ext/builder.rb:205:in `block in build_extensions'
/opt/ruby/lib/ruby/3.2.0+2/rubygems/ext/builder.rb:202:in `each'
/opt/ruby/lib/ruby/3.2.0+2/rubygems/ext/builder.rb:202:in `build_extensions'
/opt/ruby/lib/ruby/3.2.0+2/rubygems/installer.rb:843:in `build_extensions'
/opt/ruby/lib/ruby/3.2.0+2/bundler/rubygems_gem_installer.rb:72:in `build_extensions'
/opt/ruby/lib/ruby/3.2.0+2/bundler/rubygems_gem_installer.rb:28:in `install'
/opt/ruby/lib/ruby/3.2.0+2/bundler/source/rubygems.rb:202:in `install'
/opt/ruby/lib/ruby/3.2.0+2/bundler/installer/gem_installer.rb:54:in `install'
/opt/ruby/lib/ruby/3.2.0+2/bundler/installer/gem_installer.rb:16:in `install_from_spec'
/opt/ruby/lib/ruby/3.2.0+2/bundler/installer/parallel_installer.rb:186:in `do_install'
/opt/ruby/lib/ruby/3.2.0+2/bundler/installer/parallel_installer.rb:177:in `block in worker_pool'
/opt/ruby/lib/ruby/3.2.0+2/bundler/worker.rb:62:in `apply_func'
/opt/ruby/lib/ruby/3.2.0+2/bundler/worker.rb:57:in `block in process_queue'
/opt/ruby/lib/ruby/3.2.0+2/bundler/worker.rb:54:in `loop'
/opt/ruby/lib/ruby/3.2.0+2/bundler/worker.rb:54:in `process_queue'
/opt/ruby/lib/ruby/3.2.0+2/bundler/worker.rb:91:in `block (2 levels) in create_threads'
An error occurred while installing sqlite3 (1.5.4), and Bundler cannot continue.
In Gemfile:
sqlite3
The mkmf error log at that time was as follows.
root@0f6123678405:/myapp# cat /opt/ruby/lib/ruby/gems/3.2.0+2/extensions/x86_64-linux/3.2.0+2/sqlite3-1.5.4/mkmf.log
package configuration for /opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc is not found
Updated by mdalessio (Mike Dalessio) almost 2 years ago
Hi, I'm a co-maintainer of the sqlite3 gem and I wanted to add a little more context.
In v1.5, we simplified the extconf.rb
by using MakeMakefile.pkg_config
to read the sqlite3.pc
file generated earlier during installation. Prior to v1.5 we only used the pkg_config
method to find system libraries.
Updated by mdalessio (Mike Dalessio) almost 2 years ago
When I examine the files left in the docker container, I see this error in the mkmf.log
:
package configuration for /opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc is not found
and that pc file contains:
root@8da2dee444e4:/myapp# cat /opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc
# Package Information for pkg-config
prefix=/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include
Name: SQLite
Description: SQL database engine
Version: 3.40.0
Libs: -L${libdir} -lsqlite3
Libs.private: -lz -lm -ldl -lpthread
Cflags: -I${includedir}
This all seems as expected, so I don't think this is a problem with the gem. I'm not sure I understand why pkg_config
is raising an exception, so I'll continue investigating.
Updated by mdalessio (Mike Dalessio) almost 2 years ago
When sqlite3-ruby's (v1.5+) extconf calls pkg_config()
, the argument pkg
is "/opt/ruby/lib/ruby/gems/3.2.0+2/gems/sqlite3-1.5.4/ports/x86_64-linux-gnu/sqlite3/3.40.0/lib/pkgconfig/sqlite3.pc".
In this docker container, RbConfig::CONFIG["PKG_CONFIG"]
is set to ""
.
Previously I believe that this method set the variable pkgconfig
to "pkg-config". However, after 7f1ca66 it falls through the if
conditions and is set to nil
, causing this error.
Updated by nobu (Nobuyoshi Nakada) almost 2 years ago
Isn’t this same as #19189?
Updated by yahonda (Yasuo Honda) almost 2 years ago
I have opened a new one because it does not reproduce against Ruby 3.1.3 Docker images.
FROM ruby:3.1.3
FROM rubylang/ruby:3.1.3-focal
#19189 says:
Starting with Ruby 3.1.3, mkmf.rb calls to pkg-config will fail if pkg-config wasn't present on the system at the time it was built.
Updated by yahonda (Yasuo Honda) almost 2 years ago
Confirmed this issue has been fixed since 613fca01486e47dee9364a2fd86b5f5e77fe23c8
Updated by hsbt (Hiroshi SHIBATA) almost 2 years ago
- Status changed from Open to Closed
Updated by nobu (Nobuyoshi Nakada) almost 2 years ago
- Related to deleted (Bug #19189: Ruby 3.1.3/3.2.x can no longer find pkg-config if not present at buildtime)
Updated by nobu (Nobuyoshi Nakada) almost 2 years ago
- Is duplicate of Bug #19189: Ruby 3.1.3/3.2.x can no longer find pkg-config if not present at buildtime added