https://redmine.ruby-lang.org/https://redmine.ruby-lang.org/favicon.ico?17113305112011-09-20T07:35:51ZRuby Issue Tracking SystemRuby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=209002011-09-20T07:35:51Zdrbrain (Eric Hodel)drbrain@segment7.net
<ul><li><strong>File</strong> <a href="/attachments/2087">net.http.rb.ssl_session_reuse.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/2087/net.http.rb.ssl_session_reuse.patch">net.http.rb.ssl_session_reuse.patch</a> added</li></ul><p>The original patch has unrelated changes.</p> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=209012011-09-20T07:35:58Zdrbrain (Eric Hodel)drbrain@segment7.net
<ul><li><strong>File</strong> deleted (<del><i>net.http.rb.ssl_session_reuse.patch</i></del>)</li></ul> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=215302011-10-25T18:01:04Znahi (Hiroshi Nakamura)nakahiro@gmail.com
<ul></ul><p>The patch does;</p>
<ul>
<li>Let HTTP object keeps last SSLSession and reuse it when it tries to connect next time.</li>
<li>Initializes a SSLContext only at first connection.</li>
</ul>
<p>Comments;</p>
<ul>
<li>Is Net::HTTP object made for reusing? Is '5.times { http.start; http.finish }' supposed to work? (Sorry, I don't know Net::HTTP well...)</li>
<li>Even so, ssl_parameters is ignored except at first connection, even if some params can be updated after the first connection.</li>
<li>Reusing SSLSesion without checking its timeout might cause compatibility issue though I don't know if that's true or not. We need some compatibility study. Eric, you found an issue with OpenSSL 0.9.7 + Google's false-start SSL server, didn't you?</li>
</ul>
<p>Adding SSLSession to Net::HTTP is not so bad if it is supposed to work, but I think that making Net::HTTP fat could conflict with your Agent proposal <a class="issue tracker-2 status-2 priority-4 priority-default" title="Feature: HTTP user-agent class (Assigned)" href="https://redmine.ruby-lang.org/issues/5064">#5064</a>. Isn't it good just adding ssl_session property to Net::HTTP?</p> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=215312011-10-25T19:40:25Znahi (Hiroshi Nakamura)nakahiro@gmail.com
<ul></ul><p>According to this tweet, ext/openssl in ruby 1.9.1 or earlier didn't work well with Google's SSL false-start servers, right?<br>
<a href="https://twitter.com/#!/drbrain/status/115945404830449664" class="external">https://twitter.com/#!/drbrain/status/115945404830449664</a></p>
<p>Then it might be a bug that existed in old version of ext/openssl.</p> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=215632011-10-26T11:39:00Zdrbrain (Eric Hodel)drbrain@segment7.net
<ul></ul><p>=begin<br>
Net::HTTP objects can be reused. You may start and finish a connection as many times as you like (the net-http-persistent gem works this way).</p>
<p>Currently the SSLSession can only be initialized once due to OpenSSL restrictions. To change the values you must make a new Net::HTTP object, so I think changing this behavior (if it is needed) is a separate issue.</p>
<p>$ cat test.rb<br>
require 'socket'<br>
require 'openssl'</p>
<p>key = OpenSSL::PKey::RSA.new 256<br>
cert = OpenSSL::X509::Certificate.new<br>
cert.version = 2<br>
cert.serial = 0<br>
cert.not_before = Time.now<br>
cert.not_after = Time.now + 3600<br>
cert.public_key = key.public_key<br>
cert.subject = OpenSSL::X509::Name.parse 'CN=nobody/DC=example'</p>
<p>store = OpenSSL::X509::Store.new<br>
store.set_default_paths</p>
<p>socket = TCPSocket.new 'localhost', 80 # any open port will do<br>
ssl_context = OpenSSL::SSL::SSLContext.new<br>
ssl_socket = OpenSSL::SSL::SSLSocket.new socket, ssl_context</p>
<p>def try ssl_context, params<br>
ssl_context.set_params params<br>
rescue<br>
p params.keys.first => $!.message<br>
end</p>
<p>try ssl_context, :@ssl_version => 3<br>
try ssl_context, :@key => key<br>
try ssl_context, :@cert => cert<br>
try ssl_context, :@ca_file => '/nonexistent/file'<br>
try ssl_context, :@ca_path => '/nonexistent/path'<br>
try ssl_context, :@cert_store => store<br>
try ssl_context, :@ciphers => []<br>
try ssl_context, :@verify_mode => OpenSSL::SSL::VERIFY_PEER<br>
try ssl_context, :@verify_callback => proc { |x| }<br>
try ssl_context, :@verify_depth => 2<br>
try ssl_context, :@ssl_timeout => 99</p>
<p>$ make runruby<br>
./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems ./test.rb<br>
{:@ssl_version=>"can't modify frozen OpenSSL::SSL::SSLContext"}<br>
{:@key=>"can't modify frozen OpenSSL::SSL::SSLContext"}<br>
{:@cert=>"can't modify frozen OpenSSL::SSL::SSLContext"}<br>
{:@ca_file=>"can't modify frozen OpenSSL::SSL::SSLContext"}<br>
{:@ca_path=>"can't modify frozen OpenSSL::SSL::SSLContext"}<br>
{:@cert_store=>"can't modify frozen OpenSSL::SSL::SSLContext"}<br>
{:@ciphers=>"can't modify frozen OpenSSL::SSL::SSLContext"}<br>
{:@verify_mode=>"can't modify frozen OpenSSL::SSL::SSLContext"}<br>
{:@verify_callback=>"can't modify frozen OpenSSL::SSL::SSLContext"}<br>
{:@verify_depth=>"can't modify frozen OpenSSL::SSL::SSLContext"}<br>
{:@ssl_timeout=>"can't modify frozen OpenSSL::SSL::SSLContext"}<br>
$ ./miniruby -v<br>
ruby 2.0.0dev (2011-10-25 trunk 33524) [x86_64-darwin11.1.0]</p>
<p>I will update the patch to check for the timeout, I did not know it existed.</p>
<p>With this code:</p>
<p><a href="https://github.com/drbrain/net-http-persistent/blob/master/lib/net/http/persistent/ssl_reuse.rb" class="external">https://github.com/drbrain/net-http-persistent/blob/master/lib/net/http/persistent/ssl_reuse.rb</a></p>
<p>I was only able to reproduce the issue on Ruby 1.9.1, not Ruby 1.8.7, 1.9.2, 1.9.3 or ruby trunk.<br>
=end</p> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=215642011-10-26T11:49:16Zdrbrain (Eric Hodel)drbrain@segment7.net
<ul></ul><p>I think it is best to add SSL session reuse to Net::HTTP because it is very hard to add it at a higher layer without adding hooks to Net::HTTP.</p>
<p>Net::HTTP is great at making connections and reading and writing the HTTP protocol. I think a higher layer like in <a class="issue tracker-2 status-2 priority-4 priority-default" title="Feature: HTTP user-agent class (Assigned)" href="https://redmine.ruby-lang.org/issues/5064">#5064</a> should stick to implementing the features of HTTP beyond what Net::HTTP is good at.</p> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=215672011-10-26T14:58:21ZMartinBosslet (Martin Bosslet)Martin.Bosslet@gmail.com
<ul></ul><p>Eric Hodel wrote:</p>
<blockquote>
<p>I will update the patch to check for the timeout, I did not know it existed.</p>
</blockquote>
<p>I could imagine that OpenSSL itself already checks for a timeout. I'm not sure,<br>
and we have no related tests for test_ssl_session.rb. I'll look into that,<br>
regardless of the outcome I would add specific timeout tests.</p> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=215712011-10-26T22:23:16ZAnonymous
<ul></ul><p>On 10/26/2011 11:39 AM, Eric Hodel wrote:</p>
<blockquote>
<p>Net::HTTP objects can be reused. You may start and finish a<br>
connection as many times as you like (the net-http-persistent gem<br>
works this way).</p>
</blockquote>
<p>OK, so it's just I was wrong. I felt that I saw an issue for<br>
restarting Net::HTTP with #start after #finish, but I cannot find a<br>
ticket. I'm just confusing it with another issue. Either it's<br>
already fixed.</p>
<blockquote>
<p>Currently the SSLSession can only be initialized once due to<br>
OpenSSL restrictions. To change the values you must make a new<br>
Net::HTTP object, so I think changing this behavior (if it is<br>
needed) is a separate issue.</p>
</blockquote>
<p>SSLSocket --<>> SSLContext <<>-- Session<br>
<> A<br>
| |<br>
+------------------------------+</p>
<p>You can't modify SSLContext after you create SSLSocket. You must<br>
create new SSLContext for new SSLSocket if you need to update ssl<br>
params. It might conflict with semantics of Net::HTTP object, thought<br>
I don't have any idea what Net::HTTP object should know.</p>
<blockquote>
<p>I will update the patch to check for the timeout, I did not know it<br>
existed.</p>
</blockquote>
<p>OpenSSL has a client session cache in SSLContext but it's not so<br>
useful because you need to keep Sessions by yourself, and pick proper<br>
Session for each server. Here's a sample usage of client session cache:</p>
<p><a href="https://github.com/nahi/httpclient/commit/7fc04933961ea3ea5a2aa595172ca7cd29a718f5" class="external">https://github.com/nahi/httpclient/commit/7fc04933961ea3ea5a2aa595172ca7cd29a718f5</a></p>
<p>You would want to implement session cache instead.</p>
<p>FYI: In contrast to the client session cache, the server session cache<br>
is very useful and everyone should use it whenever you need a server<br>
session cache.</p>
<blockquote>
<p>With this code:</p>
<p><a href="https://github.com/drbrain/net-http-persistent/blob/master/lib/net/http/persistent/ssl_reuse.rb" class="external">https://github.com/drbrain/net-http-persistent/blob/master/lib/net/http/persistent/ssl_reuse.rb</a></p>
<p>I was only able to reproduce the issue on Ruby 1.9.1, not Ruby<br>
1.8.7, 1.9.2, 1.9.3 or ruby trunk.</p>
</blockquote>
<p>Thank you. I'll try to reproduce it.</p> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=220722011-11-10T11:23:08Zdrbrain (Eric Hodel)drbrain@segment7.net
<ul></ul><p>On Oct 26, 2011, at 6:06 AM, Hiroshi Nakamura wrote:</p>
<blockquote>
<p>On 10/26/2011 11:39 AM, Eric Hodel wrote:</p>
<blockquote>
<p>Net::HTTP objects can be reused. You may start and finish a<br>
connection as many times as you like (the net-http-persistent gem<br>
works this way).</p>
</blockquote>
<p>OK, so it's just I was wrong. I felt that I saw an issue for<br>
restarting Net::HTTP with #start after #finish, but I cannot find a<br>
ticket. I'm just confusing it with another issue. Either it's<br>
already fixed.</p>
<blockquote>
<p>Currently the SSLSession can only be initialized once due to<br>
OpenSSL restrictions. To change the values you must make a new<br>
Net::HTTP object, so I think changing this behavior (if it is<br>
needed) is a separate issue.</p>
</blockquote>
<p>SSLSocket --<>> SSLContext <<>-- Session<br>
<> A<br>
| |<br>
+------------------------------+</p>
<p>You can't modify SSLContext after you create SSLSocket. You must<br>
create new SSLContext for new SSLSocket if you need to update ssl<br>
params. It might conflict with semantics of Net::HTTP object, thought<br>
I don't have any idea what Net::HTTP object should know.</p>
</blockquote>
<p>With net/http in 1.9.3, modifying SSL parameters after Net::HTTP#start has no effect. Adding it as a new feature is not difficult, though (delete two lines). It does not cause any conflicts for Net::HTTP that I can see.</p>
<blockquote>
<blockquote>
<p>I will update the patch to check for the timeout, I did not know it<br>
existed.</p>
</blockquote>
<p>OpenSSL has a client session cache in SSLContext but it's not so<br>
useful because you need to keep Sessions by yourself, and pick proper<br>
Session for each server. Here's a sample usage of client session cache:</p>
<p><a href="https://github.com/nahi/httpclient/commit/7fc04933961ea3ea5a2aa595172ca7cd29a718f5" class="external">https://github.com/nahi/httpclient/commit/7fc04933961ea3ea5a2aa595172ca7cd29a718f5</a></p>
<p>You would want to implement session cache instead.</p>
</blockquote>
<p>I think enabling the session cache is useless for net/http because it is single-connection oriented. Instead, just using an ivar to store the session is OK.</p>
<p>In <a href="http://www.openssl.org/docs/ssl/SSL_CTX_set_session_cache_mode.html" class="external">http://www.openssl.org/docs/ssl/SSL_CTX_set_session_cache_mode.html</a>, enabling SSL_SESS_CACHE_CLIENT says:</p>
<blockquote>
<p>Client sessions are added to the session cache. As there is no reliable way for the OpenSSL library to know whether a session should be reused or which session to choose (due to the abstract BIO layer the SSL engine does not have details about the connection), the application must select the session to be reused by using the SSL_set_session(3) function. This option is not activated by default.</p>
</blockquote>
<p>I think for net/http the client session cache is useless. net/http only connects to one server per instance and will only have one context alive at a time, so the cache will not hold more than one session at a time.</p>
<p>Instead of jumping through the hoops of the client session cache (cache-managing class, callbacks) it will be easier to store the session in an instance variable after connect() and SSL negotiation (since there can only ever be one item in the cache for net/http) and apply the session from the ivar via SSL_set_session (SSLSocket#session</p> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=222532011-11-17T18:07:19ZMartinBosslet (Martin Bosslet)Martin.Bosslet@gmail.com
<ul></ul><p>Eric Hodel wrote:</p>
<blockquote>
<p>On Oct 26, 2011, at 6:06 AM, Hiroshi Nakamura wrote:</p>
<blockquote>
<p>On 10/26/2011 11:39 AM, Eric Hodel wrote:</p>
</blockquote>
</blockquote>
<blockquote>
<p>I think enabling the session cache is useless for net/http because it is single-connection oriented. Instead, just using an ivar to store the session is OK.</p>
<p>In <a href="http://www.openssl.org/docs/ssl/SSL_CTX_set_session_cache_mode.html" class="external">http://www.openssl.org/docs/ssl/SSL_CTX_set_session_cache_mode.html</a>, enabling SSL_SESS_CACHE_CLIENT says:</p>
<blockquote>
<p>Client sessions are added to the session cache. As there is no reliable way for the OpenSSL library to know whether a session should be reused or which session to choose (due to the abstract BIO layer the SSL engine does not have details about the connection), the application must select the session to be reused by using the SSL_set_session(3) function. This option is not activated by default.</p>
</blockquote>
<p>I think for net/http the client session cache is useless. net/http only connects to one server per instance and will only have one context alive at a time, so the cache will not hold more than one session at a time.</p>
<p>Instead of jumping through the hoops of the client session cache (cache-managing class, callbacks) it will be easier to store the session in an instance variable after connect() and SSL negotiation (since there can only ever be one item in the cache for net/http) and apply the session from the ivar via SSL_set_session (SSLSocket#session</p>
</blockquote>
<p>We already had discussed some of this on IRC. I looked into the TLS RFCs a couple of days back, and from the discussion and the RFC I conclude the same as Eric, that keeping the session as an instance variable should suffice.</p> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=223512011-11-23T09:28:32Zdrbrain (Eric Hodel)drbrain@segment7.net
<ul><li><strong>File</strong> <a href="/attachments/2265">net.http.rb.ssl_session_reuse.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/2265/net.http.rb.ssl_session_reuse.patch">net.http.rb.ssl_session_reuse.patch</a> added</li></ul><p>Updated patch</p> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=254282012-03-30T08:49:36Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Assigned</i></li><li><strong>Assignee</strong> set to <i>naruse (Yui NARUSE)</i></li><li><strong>Target version</strong> changed from <i>1.9.4</i> to <i>2.0.0</i></li></ul><p>I tentatively assign this issue to Naruse-san because<br>
he is running for the maintainer of net/http.</p>
<p>--<br>
Yusuke Endoh <a href="mailto:mame@tsg.ne.jp" class="email">mame@tsg.ne.jp</a></p> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=254372012-03-30T11:23:17ZAnonymous
<ul><li><strong>File</strong> <a href="/attachments/2549">noname</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/2549/noname">noname</a> added</li></ul><p>On Fri, Mar 30, 2012 at 08:49:36AM +0900, mame (Yusuke Endoh) wrote:</p>
<blockquote>
<p>Issue <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Add SSL session reuse to Net::HTTP (Closed)" href="https://redmine.ruby-lang.org/issues/5341">#5341</a> has been updated by mame (Yusuke Endoh).</p>
<p>Status changed from Open to Assigned<br>
Assignee set to naruse (Yui NARUSE)<br>
Target version changed from 1.9.4 to 2.0.0</p>
<p>I tentatively assign this issue to Naruse-san because<br>
he is running for the maintainer of net/http.</p>
</blockquote>
<p>When will elections be held? ;-)</p>
<p>--<br>
Aaron Patterson<br>
<a href="http://tenderlovemaking.com/" class="external">http://tenderlovemaking.com/</a></p> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=281882012-07-18T13:21:32Znaruse (Yui NARUSE)naruse@airemix.jp
<ul><li><strong>Assignee</strong> changed from <i>naruse (Yui NARUSE)</i> to <i>drbrain (Eric Hodel)</i></li></ul><p>drbrain (Eric Hodel) wrote:</p>
<blockquote>
<p>Updated patch</p>
</blockquote>
<p>If Martin checked this patch, I'm ok; commit it.</p> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=281962012-07-19T00:54:06ZMartinBosslet (Martin Bosslet)Martin.Bosslet@gmail.com
<ul></ul><p>Eric, nahi and I discussed this a while ago and I think we all agreed finally? I remember I was OK with it :)</p> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=284142012-07-25T09:06:01Zdrbrain (Eric Hodel)drbrain@segment7.net
<ul><li><strong>Status</strong> changed from <i>Assigned</i> to <i>Closed</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li></ul><p>This issue was solved with changeset r36528.<br>
Eric, thank you for reporting this issue.<br>
Your contribution to Ruby is greatly appreciated.<br>
May Ruby be with you.</p>
<hr>
<ul>
<li>lib/net/http.rb: Added SSL session reuse across connections for a<br>
single instance to speed up connection. [Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Add SSL session reuse to Net::HTTP (Closed)" href="https://redmine.ruby-lang.org/issues/5341">#5341</a>]</li>
<li>NEWS: ditto</li>
<li>test/net/http/test_https.rb: Tests for <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Add SSL session reuse to Net::HTTP (Closed)" href="https://redmine.ruby-lang.org/issues/5341">#5341</a></li>
</ul> Ruby master - Feature #5341: Add SSL session reuse to Net::HTTPhttps://redmine.ruby-lang.org/issues/5341?journal_id=284392012-07-25T22:53:20Znahi (Hiroshi Nakamura)nakahiro@gmail.com
<ul></ul><p>Sorry for late reply.</p>
<p>On Thu, Nov 10, 2011 at 11:04 AM, Eric Hodel <a href="mailto:drbrain@segment7.net" class="email">drbrain@segment7.net</a> wrote:</p>
<blockquote>
<blockquote>
<blockquote>
<p>I will update the patch to check for the timeout, I did not know it<br>
existed.</p>
</blockquote>
<p>OpenSSL has a client session cache in SSLContext but it's not so<br>
useful because you need to keep Sessions by yourself, and pick proper<br>
Session for each server. Here's a sample usage of client session cache:</p>
<p><a href="https://github.com/nahi/httpclient/commit/7fc04933961ea3ea5a2aa595172ca7cd29a718f5" class="external">https://github.com/nahi/httpclient/commit/7fc04933961ea3ea5a2aa595172ca7cd29a718f5</a></p>
<p>You would want to implement session cache instead.</p>
</blockquote>
<p>I think enabling the session cache is useless for net/http because it is single-connection oriented. Instead, just using an ivar to store the session is OK.</p>
<p>In <a href="http://www.openssl.org/docs/ssl/SSL_CTX_set_session_cache_mode.html" class="external">http://www.openssl.org/docs/ssl/SSL_CTX_set_session_cache_mode.html</a>, enabling SSL_SESS_CACHE_CLIENT says:</p>
<blockquote>
<p>Client sessions are added to the session cache. As there is no reliable way for the OpenSSL library to know whether a session should be reused or which session to choose (due to the abstract BIO layer the SSL engine does not have details about the connection), the application must select the session to be reused by using the SSL_set_session(3) function. This option is not activated by default.</p>
</blockquote>
<p>I think for net/http the client session cache is useless. net/http only connects to one server per instance and will only have one context alive at a time, so the cache will not hold more than one session at a time.</p>
</blockquote>
<p>Indeed. That's insufficient.</p>
<blockquote>
<p>Instead of jumping through the hoops of the client session cache (cache-managing class, callbacks) it will be easier to store the session in an instance variable after connect() and SSL negotiation (since there can only ever be one item in the cache for net/http) and apply the session from the ivar via SSL_set_session (SSLSocket#session=) when we call connect() again.</p>
</blockquote>
<p>I like the new patch. Let's commit it and see how it affects existing servers.</p>
<p>Thank you!</p>