Backport #8240: SSLSocket breaks other connections or files on GC

• Status: Closed
• Assignee: nagachika (Tomoyuki Chikanaga)

[shugo (Shugo Maeda)]

When an OpenSSL::SSL::SSLSocket is recycled by GC, SSL_shutdown() is called,
and SSL_shutdown() sends a close-notify alert message.
However at the GC time, the original socket might have already been closed,
and thus its file descriptor might be reused for another socket or file.

This problem can be reproduced as follows:

$ cat t.rb
require "socket"
require "openssl"

loop do
sock = TCPSocket.new("localhost", 443)
GC.start
ssl = OpenSSL::SSL::SSLSocket.new(sock)
ssl.connect
sock.close
end
$ ruby -v t.rb
ruby 2.1.0dev (2013-04-08 trunk 40183) [i686-linux]
t.rb:8:in connect': SSL_connect SYSCALL returned=5 errno=0 state=unknown state (OpenSSL::SSL::SSLError) from t.rb:8:in block in '
from t.rb:4:in loop' from t.rb:4:in '

An SSLError is raised because a close-notify alert message is sent to the server by GC
instead of a client hello message.
If the file descriptor is reused for a file, not a socket, the file would get broken.
This problem occurs rarely, but its impact is very serious.

IMHO, the free function of a DATA object should not do any task other than resource release.

Furthermore, SSLSocket#close calls SSL_shutdown(), but the original socket might have been closed,
in which case SSL_shutdown() (and @io.close) should not be called either.

The attached patch fixes these problems.

[bpot (Bob Potter)]

Cool! This should also resolve #7584

• Status changed from Assigned to Closed
• % Done changed from 0 to 100

This issue was solved with changeset r40304.
Shugo, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

---

• ext/openssl/ossl_ssl.c: Correct shutdown behavior w.r.t GC.

• test/openssl/test_ssl.rb: Add tests to verify correct behavior.

[Bug #8240] Patch provided by Shugo Maeda. Thanks!

[nagachika (Tomoyuki Chikanaga)]

• Tracker changed from Bug to Backport
• Project changed from Ruby master to Backport200
• Category deleted (ext)
• Status changed from Closed to Assigned
• Assignee changed from MartinBosslet (Martin Bosslet) to nagachika (Tomoyuki Chikanaga)
• Target version deleted (2.1.0)

[nagachika (Tomoyuki Chikanaga)]

• Status changed from Assigned to Closed

This issue was solved with changeset r40387.
Shugo, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

---

merge revision(s) 40304: [Backport #8240]

* ext/openssl/ossl_ssl.c: Correct shutdown behavior w.r.t GC.

* test/openssl/test_ssl.rb: Add tests to verify correct behavior.
[Bug #8240] Patch provided by Shugo Maeda. Thanks!