Project

General

Profile

Actions

Feature #5341

closed

Add SSL session reuse to Net::HTTP

Added by drbrain (Eric Hodel) over 12 years ago. Updated over 11 years ago.

Status:
Closed
Target version:
[ruby-core:39629]

Description

SSL session reuse allows reconnection to an HTTPS server to avoid an SSL handshake which avoids extra computations and network round-trips and increases the performance of SSL connections.


Files

net.http.rb.ssl_session_reuse.patch (2.21 KB) net.http.rb.ssl_session_reuse.patch Corrected patch drbrain (Eric Hodel), 09/20/2011 07:35 AM
net.http.rb.ssl_session_reuse.patch (1.99 KB) net.http.rb.ssl_session_reuse.patch Create a new ssl_context for each call to #connect drbrain (Eric Hodel), 11/23/2011 09:28 AM
noname (500 Bytes) noname Anonymous, 03/30/2012 11:23 AM
Actions #2

Updated by drbrain (Eric Hodel) over 12 years ago

  • File deleted (net.http.rb.ssl_session_reuse.patch)

Updated by nahi (Hiroshi Nakamura) over 12 years ago

The patch does;

  • Let HTTP object keeps last SSLSession and reuse it when it tries to connect next time.
  • Initializes a SSLContext only at first connection.

Comments;

  • Is Net::HTTP object made for reusing? Is '5.times { http.start; http.finish }' supposed to work? (Sorry, I don't know Net::HTTP well...)
  • Even so, ssl_parameters is ignored except at first connection, even if some params can be updated after the first connection.
  • Reusing SSLSesion without checking its timeout might cause compatibility issue though I don't know if that's true or not. We need some compatibility study. Eric, you found an issue with OpenSSL 0.9.7 + Google's false-start SSL server, didn't you?

Adding SSLSession to Net::HTTP is not so bad if it is supposed to work, but I think that making Net::HTTP fat could conflict with your Agent proposal #5064. Isn't it good just adding ssl_session property to Net::HTTP?

Updated by nahi (Hiroshi Nakamura) over 12 years ago

According to this tweet, ext/openssl in ruby 1.9.1 or earlier didn't work well with Google's SSL false-start servers, right?
https://twitter.com/#!/drbrain/status/115945404830449664

Then it might be a bug that existed in old version of ext/openssl.

Updated by drbrain (Eric Hodel) over 12 years ago

=begin
Net::HTTP objects can be reused. You may start and finish a connection as many times as you like (the net-http-persistent gem works this way).

Currently the SSLSession can only be initialized once due to OpenSSL restrictions. To change the values you must make a new Net::HTTP object, so I think changing this behavior (if it is needed) is a separate issue.

$ cat test.rb
require 'socket'
require 'openssl'

key = OpenSSL::PKey::RSA.new 256
cert = OpenSSL::X509::Certificate.new
cert.version = 2
cert.serial = 0
cert.not_before = Time.now
cert.not_after = Time.now + 3600
cert.public_key = key.public_key
cert.subject = OpenSSL::X509::Name.parse 'CN=nobody/DC=example'

store = OpenSSL::X509::Store.new
store.set_default_paths

socket = TCPSocket.new 'localhost', 80 # any open port will do
ssl_context = OpenSSL::SSL::SSLContext.new
ssl_socket = OpenSSL::SSL::SSLSocket.new socket, ssl_context

def try ssl_context, params
ssl_context.set_params params
rescue
p params.keys.first => $!.message
end

try ssl_context, :@ssl_version => 3
try ssl_context, :@key => key
try ssl_context, :@cert => cert
try ssl_context, :@ca_file => '/nonexistent/file'
try ssl_context, :@ca_path => '/nonexistent/path'
try ssl_context, :@cert_store => store
try ssl_context, :@ciphers => []
try ssl_context, :@verify_mode => OpenSSL::SSL::VERIFY_PEER
try ssl_context, :@verify_callback => proc { |x| }
try ssl_context, :@verify_depth => 2
try ssl_context, :@ssl_timeout => 99

$ make runruby
./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems ./test.rb
{:@ssl_version=>"can't modify frozen OpenSSL::SSL::SSLContext"}
{:@key=>"can't modify frozen OpenSSL::SSL::SSLContext"}
{:@cert=>"can't modify frozen OpenSSL::SSL::SSLContext"}
{:@ca_file=>"can't modify frozen OpenSSL::SSL::SSLContext"}
{:@ca_path=>"can't modify frozen OpenSSL::SSL::SSLContext"}
{:@cert_store=>"can't modify frozen OpenSSL::SSL::SSLContext"}
{:@ciphers=>"can't modify frozen OpenSSL::SSL::SSLContext"}
{:@verify_mode=>"can't modify frozen OpenSSL::SSL::SSLContext"}
{:@verify_callback=>"can't modify frozen OpenSSL::SSL::SSLContext"}
{:@verify_depth=>"can't modify frozen OpenSSL::SSL::SSLContext"}
{:@ssl_timeout=>"can't modify frozen OpenSSL::SSL::SSLContext"}
$ ./miniruby -v
ruby 2.0.0dev (2011-10-25 trunk 33524) [x86_64-darwin11.1.0]

I will update the patch to check for the timeout, I did not know it existed.

With this code:

https://github.com/drbrain/net-http-persistent/blob/master/lib/net/http/persistent/ssl_reuse.rb

I was only able to reproduce the issue on Ruby 1.9.1, not Ruby 1.8.7, 1.9.2, 1.9.3 or ruby trunk.
=end

Updated by drbrain (Eric Hodel) over 12 years ago

I think it is best to add SSL session reuse to Net::HTTP because it is very hard to add it at a higher layer without adding hooks to Net::HTTP.

Net::HTTP is great at making connections and reading and writing the HTTP protocol. I think a higher layer like in #5064 should stick to implementing the features of HTTP beyond what Net::HTTP is good at.

Updated by MartinBosslet (Martin Bosslet) over 12 years ago

Eric Hodel wrote:

I will update the patch to check for the timeout, I did not know it existed.

I could imagine that OpenSSL itself already checks for a timeout. I'm not sure,
and we have no related tests for test_ssl_session.rb. I'll look into that,
regardless of the outcome I would add specific timeout tests.

Updated by Anonymous over 12 years ago

On 10/26/2011 11:39 AM, Eric Hodel wrote:

Net::HTTP objects can be reused. You may start and finish a
connection as many times as you like (the net-http-persistent gem
works this way).

OK, so it's just I was wrong. I felt that I saw an issue for
restarting Net::HTTP with #start after #finish, but I cannot find a
ticket. I'm just confusing it with another issue. Either it's
already fixed.

Currently the SSLSession can only be initialized once due to
OpenSSL restrictions. To change the values you must make a new
Net::HTTP object, so I think changing this behavior (if it is
needed) is a separate issue.

SSLSocket --<>> SSLContext <<>-- Session
<> A
| |
+------------------------------+

You can't modify SSLContext after you create SSLSocket. You must
create new SSLContext for new SSLSocket if you need to update ssl
params. It might conflict with semantics of Net::HTTP object, thought
I don't have any idea what Net::HTTP object should know.

I will update the patch to check for the timeout, I did not know it
existed.

OpenSSL has a client session cache in SSLContext but it's not so
useful because you need to keep Sessions by yourself, and pick proper
Session for each server. Here's a sample usage of client session cache:

https://github.com/nahi/httpclient/commit/7fc04933961ea3ea5a2aa595172ca7cd29a718f5

You would want to implement session cache instead.

FYI: In contrast to the client session cache, the server session cache
is very useful and everyone should use it whenever you need a server
session cache.

With this code:

https://github.com/drbrain/net-http-persistent/blob/master/lib/net/http/persistent/ssl_reuse.rb

I was only able to reproduce the issue on Ruby 1.9.1, not Ruby
1.8.7, 1.9.2, 1.9.3 or ruby trunk.

Thank you. I'll try to reproduce it.

Updated by drbrain (Eric Hodel) over 12 years ago

On Oct 26, 2011, at 6:06 AM, Hiroshi Nakamura wrote:

On 10/26/2011 11:39 AM, Eric Hodel wrote:

Net::HTTP objects can be reused. You may start and finish a
connection as many times as you like (the net-http-persistent gem
works this way).

OK, so it's just I was wrong. I felt that I saw an issue for
restarting Net::HTTP with #start after #finish, but I cannot find a
ticket. I'm just confusing it with another issue. Either it's
already fixed.

Currently the SSLSession can only be initialized once due to
OpenSSL restrictions. To change the values you must make a new
Net::HTTP object, so I think changing this behavior (if it is
needed) is a separate issue.

SSLSocket --<>> SSLContext <<>-- Session
<> A
| |
+------------------------------+

You can't modify SSLContext after you create SSLSocket. You must
create new SSLContext for new SSLSocket if you need to update ssl
params. It might conflict with semantics of Net::HTTP object, thought
I don't have any idea what Net::HTTP object should know.

With net/http in 1.9.3, modifying SSL parameters after Net::HTTP#start has no effect. Adding it as a new feature is not difficult, though (delete two lines). It does not cause any conflicts for Net::HTTP that I can see.

I will update the patch to check for the timeout, I did not know it
existed.

OpenSSL has a client session cache in SSLContext but it's not so
useful because you need to keep Sessions by yourself, and pick proper
Session for each server. Here's a sample usage of client session cache:

https://github.com/nahi/httpclient/commit/7fc04933961ea3ea5a2aa595172ca7cd29a718f5

You would want to implement session cache instead.

I think enabling the session cache is useless for net/http because it is single-connection oriented. Instead, just using an ivar to store the session is OK.

In http://www.openssl.org/docs/ssl/SSL_CTX_set_session_cache_mode.html, enabling SSL_SESS_CACHE_CLIENT says:

Client sessions are added to the session cache. As there is no reliable way for the OpenSSL library to know whether a session should be reused or which session to choose (due to the abstract BIO layer the SSL engine does not have details about the connection), the application must select the session to be reused by using the SSL_set_session(3) function. This option is not activated by default.

I think for net/http the client session cache is useless. net/http only connects to one server per instance and will only have one context alive at a time, so the cache will not hold more than one session at a time.

Instead of jumping through the hoops of the client session cache (cache-managing class, callbacks) it will be easier to store the session in an instance variable after connect() and SSL negotiation (since there can only ever be one item in the cache for net/http) and apply the session from the ivar via SSL_set_session (SSLSocket#session

Updated by MartinBosslet (Martin Bosslet) over 12 years ago

Eric Hodel wrote:

On Oct 26, 2011, at 6:06 AM, Hiroshi Nakamura wrote:

On 10/26/2011 11:39 AM, Eric Hodel wrote:

I think enabling the session cache is useless for net/http because it is single-connection oriented. Instead, just using an ivar to store the session is OK.

In http://www.openssl.org/docs/ssl/SSL_CTX_set_session_cache_mode.html, enabling SSL_SESS_CACHE_CLIENT says:

Client sessions are added to the session cache. As there is no reliable way for the OpenSSL library to know whether a session should be reused or which session to choose (due to the abstract BIO layer the SSL engine does not have details about the connection), the application must select the session to be reused by using the SSL_set_session(3) function. This option is not activated by default.

I think for net/http the client session cache is useless. net/http only connects to one server per instance and will only have one context alive at a time, so the cache will not hold more than one session at a time.

Instead of jumping through the hoops of the client session cache (cache-managing class, callbacks) it will be easier to store the session in an instance variable after connect() and SSL negotiation (since there can only ever be one item in the cache for net/http) and apply the session from the ivar via SSL_set_session (SSLSocket#session

We already had discussed some of this on IRC. I looked into the TLS RFCs a couple of days back, and from the discussion and the RFC I conclude the same as Eric, that keeping the session as an instance variable should suffice.

Updated by mame (Yusuke Endoh) almost 12 years ago

  • Status changed from Open to Assigned
  • Assignee set to naruse (Yui NARUSE)
  • Target version changed from 1.9.4 to 2.0.0

I tentatively assign this issue to Naruse-san because
he is running for the maintainer of net/http.

--
Yusuke Endoh

Updated by Anonymous almost 12 years ago

On Fri, Mar 30, 2012 at 08:49:36AM +0900, mame (Yusuke Endoh) wrote:

Issue #5341 has been updated by mame (Yusuke Endoh).

Status changed from Open to Assigned
Assignee set to naruse (Yui NARUSE)
Target version changed from 1.9.4 to 2.0.0

I tentatively assign this issue to Naruse-san because
he is running for the maintainer of net/http.

When will elections be held? ;-)

--
Aaron Patterson
http://tenderlovemaking.com/

Updated by naruse (Yui NARUSE) over 11 years ago

  • Assignee changed from naruse (Yui NARUSE) to drbrain (Eric Hodel)

drbrain (Eric Hodel) wrote:

Updated patch

If Martin checked this patch, I'm ok; commit it.

Updated by MartinBosslet (Martin Bosslet) over 11 years ago

Eric, nahi and I discussed this a while ago and I think we all agreed finally? I remember I was OK with it :)

Actions #16

Updated by drbrain (Eric Hodel) over 11 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r36528.
Eric, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


  • lib/net/http.rb: Added SSL session reuse across connections for a
    single instance to speed up connection. [Feature #5341]
  • NEWS: ditto
  • test/net/http/test_https.rb: Tests for #5341

Updated by nahi (Hiroshi Nakamura) over 11 years ago

Sorry for late reply.

On Thu, Nov 10, 2011 at 11:04 AM, Eric Hodel wrote:

I will update the patch to check for the timeout, I did not know it
existed.

OpenSSL has a client session cache in SSLContext but it's not so
useful because you need to keep Sessions by yourself, and pick proper
Session for each server. Here's a sample usage of client session cache:

https://github.com/nahi/httpclient/commit/7fc04933961ea3ea5a2aa595172ca7cd29a718f5

You would want to implement session cache instead.

I think enabling the session cache is useless for net/http because it is single-connection oriented. Instead, just using an ivar to store the session is OK.

In http://www.openssl.org/docs/ssl/SSL_CTX_set_session_cache_mode.html, enabling SSL_SESS_CACHE_CLIENT says:

Client sessions are added to the session cache. As there is no reliable way for the OpenSSL library to know whether a session should be reused or which session to choose (due to the abstract BIO layer the SSL engine does not have details about the connection), the application must select the session to be reused by using the SSL_set_session(3) function. This option is not activated by default.

I think for net/http the client session cache is useless. net/http only connects to one server per instance and will only have one context alive at a time, so the cache will not hold more than one session at a time.

Indeed. That's insufficient.

Instead of jumping through the hoops of the client session cache (cache-managing class, callbacks) it will be easier to store the session in an instance variable after connect() and SSL negotiation (since there can only ever be one item in the cache for net/http) and apply the session from the ivar via SSL_set_session (SSLSocket#session=) when we call connect() again.

I like the new patch. Let's commit it and see how it affects existing servers.

Thank you!

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0