Bug #13080
closed[mingw regression] broken openssl extension in 2.4.0p0
Added by jonforums (Jon Forums) over 7 years ago. Updated over 7 years ago.
Description
When building using slightly modified rubyinstaller build recipes and a msys2/mingw64-6.2.0 (32bit) toolchain, the build fails to create the openssl extension when using openssl 1.0.2j or 1.1.0c.
$ ruby --version
ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
$ ruby -ropenssl -e "puts OpenSSL::OPENSSL_VERSION"
C:/Users/Jon/Documents/RubyDev/ri-git/sandbox/ruby21_mingw/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- openssl (LoadError)
from C:/Users/Jon/Documents/RubyDev/ri-git/sandbox/ruby21_mingw/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
Attached are the failing openssl mkmf.log and Makefile files from the 2.4.0p0 build.
I can provide good versions of the files from a 2.3.3p224 x32 build using openssl 1.0.2j if needed.
Files
bad_mkmf_2.4.log (21.9 KB) bad_mkmf_2.4.log | jonforums (Jon Forums), 12/27/2016 07:46 PM | ||
bad_Makefile_2.4 (6.22 KB) bad_Makefile_2.4 | jonforums (Jon Forums), 12/27/2016 07:46 PM |
Updated by nobu (Nobuyoshi Nakada) over 7 years ago
From https://bugs.ruby-lang.org/attachments/6316/bad_mkmf_2.4.log#L229, your ssl library has references to GDI32.dll. I'm not sure why rand_win.c needs it though.
Probably you need --with-openssl-config
option for a proper pkg-config file.
Updated by jonforums (Jon Forums) over 7 years ago
Ok. I'll play with my openssl build script https://github.com/jonforums/buildlets/blob/master/build_openssl.ps1#L62 to see if I make it work.
Strange that the same openssl 1.0.2j library works fine with 2.3.3 but not 2.4.0.
Updated by jonforums (Jon Forums) over 7 years ago
The bad news is that my custom openssl 1.0.2j library and the ones built by the rubyinstaller project (32 and 64bit) both have pkg-config info like the following. Openssl 1.1.0c (32 and 64bit) built with mingw64-6.2.0 also have the -lgdi32
dep.
#file: libcrypto.pc
...
Name: OpenSSL-libcrypto
Description: OpenSSL cryptography library
Version: 1.0.2j
Requires:
Libs: -L${libdir} -lcrypto
Libs.private: -lws2_32 -lgdi32 -lcrypt32
Cflags: -I${includedir}
#file: libssl.pc
...
Name: OpenSSL-libssl
Description: Secure Sockets Layer and cryptography libraries
Version: 1.0.2j
Requires.private: libcrypto
Libs: -L${libdir} -lssl
Libs.private: -lws2_32 -lgdi32 -lcrypt32
Cflags: -I${includedir}
petudio https://winitor.com/ show that gdi32.dll is used by my INSTALL_DIR/lib/ruby/2.3.0/i386-mingw32/openssl.so
file for calling these Windows API's:
- CreateCompatibleBitmap
- DeleteObject
- GetDlBits
- GetDeviceCaps
- GetObjectA
I don't yet know why openssl 1.0.2j needs these gdi32.dll functions, and I haven't tweaked my openssl build config to see if the gdi32.dll dep can be removed. I also haven't diffed ruby's openssl 2.3.0 and 2.4.0 build config to see what changed.
That said, I doubt I can make the gdi32.dll dep go away.
Anything you see in 2.4.0's ruby openssl config that would cause both 1.0.2j and 1.1.0c to stop building the extension?
Updated by rhenium (Kazuki Yamaguchi) over 7 years ago
Apparently my change broke it:
https://github.com/ruby/openssl/commit/5ba12a364fa5019f0e0f1e356087c001330097f1
I never thought OpenSSL would actually use gdi32.dll, but it does! RAND_screen() which used those gdi32.dll functions was removed in OpenSSL 1.1.0, but it now fails because crypt32.dll is missing.
Please apply this patch, or use pkg-config, or use shared libraries of OpenSSL (which I recommend).
--- a/ext/openssl/extconf.rb
+++ b/ext/openssl/extconf.rb
@@ -37,6 +37,12 @@
Logging::message "=== Checking for required stuff... ===\n"
result = pkg_config("openssl") && have_header("openssl/ssl.h")
unless result
+ if $mswin || $mingw
+ # required for static OpenSSL libraries
+ have_library("gdi32") # OpenSSL <= 1.0.2 (for RAND_screen())
+ have_library("crypt32")
+ end
+
result = have_header("openssl/ssl.h")
result &&= %w[crypto libeay32].any? {|lib| have_library(lib, "CRYPTO_malloc")}
result &&= %w[ssl ssleay32].any? {|lib| have_library(lib, "SSL_new")}
Updated by jonforums (Jon Forums) over 7 years ago
Yes, that was very surprising to find windows openssl using gdi32.dll.
I've got more testing to do, but your patch appears to work for both 1.0.2j and 1.1.0c on 2.4.0p0:
Thank you.
$ ruby --version
ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
$ ruby -ropenssl -e "puts OpenSSL::OPENSSL_VERSION"
OpenSSL 1.1.0c 10 Nov 2016
...
$ ruby --version
ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
$ ruby -ropenssl -e "puts OpenSSL::OPENSSL_VERSION"
OpenSSL 1.0.2j 26 Sep 2016
...
$ make test
...
gcc.exe (i686-win32-sjlj-rev1, Built by MinGW-W64 project) 6.2.0
...
PASS all 1010 tests
exec ./miniruby.exe -I../../../ruby-git/lib -I. -I.ext/common ../../../ruby-git/tool/runruby.rb --extout=.ext -- --disable-gems "../../../ruby-git/bootstraptest/runner.rb" --ruby="ruby.exe --disable-gems" ../../../ruby-git/KNOWNBUGS.rb
2016-12-30 13:06:35 -0500
Driver is ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
Target is ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
KNOWNBUGS.rb PASS 0
No tests, no problem
C:/Users/Jon/Documents/RubyDev/ruby-git/basictest/test.rb:gc OK 4 27K 1210 62
test succeeded
Are you planning to commit this patch to trunk and have it backported to ruby_2_4?
Updated by jonforums (Jon Forums) over 7 years ago
Before you decide whether to commit your patch to support using static openssl libs, I'll double check that a 32bit build with the rubyinstaller-built openssl lib from https://bintray.com/oneclick/OpenKnapsack/openssl/1.0.2j#files also works.
Rubyinstaller builds shared openssl libs
https://github.com/oneclick/knapsack-recipes/blob/master/openssl/openssl-1.0.2j.knapfile#L31
while I'm using static openssl libs to mininize potential {libeay32,ssleay32}.dll windows PATH
conflicts.
I think your current extconf.rb patch will work fine with both, but will double check.
No reason to create a New Year's surprise for the rubyinstaller team.
Updated by jonforums (Jon Forums) over 7 years ago
Your patch also works when using the rubyinstaller-built shared openssl 1.0.2j lib.
make test-all
segfaults, but test segfaults and other unresolved testing failures are typical when building ruby on windows.
$ ruby --version
ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
$ ruby -ropenssl -e "puts OpenSSL::OPENSSL_VERSION"
OpenSSL 1.0.2j 26 Sep 2016
...
$ make test
...
PASS all 1010 tests
...
2016-12-30 21:50:05 -0500
Driver is ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
Target is ruby 2.4.0p0 (2016-12-24 revision 57163) [i386-mingw32]
KNOWNBUGS.rb PASS 0
No tests, no problem
C:/Users/Jon/Documents/RubyDev/ruby-git/basictest/test.rb:gc OK 4 27K 1210 62
test succeeded
Your patch gives the user the option of building whatever openssl ext setup works best for their environment, and doesn't appear to break the rubyinstaller builds.
Thanks.
Updated by Anonymous over 7 years ago
- Status changed from Open to Closed
Applied in changeset r57482.
openssl: import v2.0.3
Import Ruby/OpenSSL 2.0.3. Only bugfixes. The full commit log since
2.0.2 (imported at r57146) can be found at:
https://github.com/ruby/openssl/compare/v2.0.2...v2.0.3
Corey Bonnell (1):
Fix for ASN1::Constructive 'each' implementation
Kazuki Yamaguchi (10):
Fix build with static OpenSSL libraries on Windows
([ruby-core:78878] [Bug #13080])
Merge pull request #96 from CBonnell/master
Merge branch 'topic/windows-static-linking-without-pkg-config' into maint
appveyor.yml: update OpenSSL version to 1.0.2j
buffering: fix typo in doc
test/envutil: fix assert_raise_with_message
x509: fix OpenSSL::X509::Name#eql?
([ruby-core:79310] [Bug #13170])
ruby-openssl-docker: update versions of Ruby and OpenSSL
.travis.yml: test with Ruby 2.4
Ruby/OpenSSL 2.0.3
Updated by rhenium (Kazuki Yamaguchi) over 7 years ago
- Backport changed from 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN to 2.2: DONTNEED, 2.3: DONTNEED, 2.4: REQUIRED
The patch is committed to ruby/openssl:
https://github.com/ruby/openssl/commit/a2dc925ac646f30e7d518158d7931ff422444ffe
and imported to Ruby tree at r57482.
2.4 branch maintainer: Could you backport r57482? It fixes this issue, #13170 and a GitHub issue ruby/openssl#96.
Updated by naruse (Yui NARUSE) over 7 years ago
- Backport changed from 2.2: DONTNEED, 2.3: DONTNEED, 2.4: REQUIRED to 2.2: DONTNEED, 2.3: DONTNEED, 2.4: DONE
ruby_2_4 r57881 merged revision(s) 57482.
Updated by rhenium (Kazuki Yamaguchi) almost 7 years ago
- Related to Bug #9817: The extconf.rb for OpenSSL assumes MingW added