Bug #9714: Ruby configured to call OpenSSL engine API when it shouldn't

• Status: Rejected
• Assignee: -
• Target version: -
• ruby -v: ruby 2.1.1p76 (2014-02-24 revision 45161) [x86_64-darwin13.0]
• Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN

[steakknife (Barry Allard)]

(This issue is NOT related to the Heartbleed vulnerability.)

There is an issue compiling under OSX where system dependencies and homebrew dependencies get mixed together, resulting in an incorrectly configured Ruby which fails at runtime.

The attached patch to ext/openssl/ossl.h checks the actual configuration from openssl/ssl.h:

#if defined(HAVE_OPENSSL_ENGINE_H) && defined(HAVE_ST_ENGINE) && !defined(OPENSSL_NO_ENGINE)

instead of just

#if defined(HAVE_OPENSSL_ENGINE_H) && defined(HAVE_ST_ENGINE)

Patches

(Patches also available at https://gist.github.com/10092587)

Testing

I've tested it on several ruby 1.9.x - 2.1.1 rebuilt from source, and it seems to Do The Right Thing (tm).

All Rubies fail in a similar manner, which was expected.

Environment

Ruby 2.1.1 configure options:

--with-openssl-dir=/usr/local/opt/openssl --disable-install-doc --enable-shared

OpenSSL 1.0.1g Configure options:

zlib-dynamic shared enable-cms no-hw no-rdrand no-sctp no-md4 no-mdc2 no-rc4 no-fips no-engine

Symptoms

0 ~ ❯❯❯ gem list a -r

*** REMOTE GEMS ***

/usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require': dlopen(/usr/local/ruby/ruby-2.1.1/lib/ruby/2.1.0/x86_64-darwin13.0/openssl.bundle, 9): Symbol not found: _ENGINE_free (LoadError)
  Referenced from: /usr/local/ruby/ruby-2.1.1/lib/ruby/2.1.0/x86_64-darwin13.0/openssl.bundle
  Expected in: /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib
 in /usr/local/ruby/ruby-2.1.1/lib/ruby/2.1.0/x86_64-darwin13.0/openssl.bundle - /usr/local/ruby/ruby-2.1.1/lib/ruby/2.1.0/x86_64-darwin13.0/openssl.bundle
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/2.1.0/openssl.rb:17:in `<top (required)>'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/2.1.0/net/https.rb:22:in `<top (required)>'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/request.rb:39:in `configure_connection_for_https'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/request.rb:93:in `connection_for'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/request.rb:122:in `fetch'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/remote_fetcher.rb:337:in `request'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/remote_fetcher.rb:231:in `fetch_http'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/remote_fetcher.rb:266:in `fetch_path'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/remote_fetcher.rb:296:in `cache_update_path'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/source.rb:177:in `load_specs'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/spec_fetcher.rb:266:in `tuples_for'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/spec_fetcher.rb:226:in `block in available_specs'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/source_list.rb:97:in `each'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/source_list.rb:97:in `each_source'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/spec_fetcher.rb:222:in `available_specs'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/spec_fetcher.rb:147:in `detect'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/commands/query_command.rb:167:in `show_gems'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/commands/query_command.rb:109:in `block in execute'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/commands/query_command.rb:109:in `each'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/commands/query_command.rb:109:in `execute'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/command.rb:305:in `invoke_with_build_args'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/command_manager.rb:167:in `process_args'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/command_manager.rb:137:in `run'
	from /usr/local/ruby/ruby-2.1.1/lib/ruby/site_ruby/2.1.0/rubygems/gem_runner.rb:54:in `run'
	from /usr/local/ruby/ruby-2.1.1/bin/gem:21:in `<main>'
1 ~ ❯❯❯

[zzak (zzak _)]

• Assignee set to 7150

[rhenium (Kazuki Yamaguchi)]

• Status changed from Open to Rejected

Indeed we should check for OPENSSL_NO_ENGINE instead of the existence of openssl/engine.h, and it's actually done in trunk (r55162).

However it is strange that this causes a runtime error. The existence of the header is checked by the have_header defined in mkmf, which sees if a code like this can be successfully preprocessed or not:

#include "ruby.h"

#include <openssl/engine.h>

This should work even if openssl/engine.h erroneously exists because openssl/engine.h checks that OPENSSL_NO_ENGINE is not defined, otherwise causes a fatal error using #error directive.