Bug #9743: memory leak in openssl ossl_pkey_verify leaks memory

• Status: Closed
• Assignee: zzak (zzak _)
• Target version: -
• ruby -v: 2.2.0
• Backport: 1.9.3: REQUIRED, 2.0.0: DONE, 2.1: DONE

[tux (Joel Westerberg)]

repeated calls to pub_key.verify(digest, signature, data) leaks memory.

from what I can gather from the openssl documentation, there seems to be a missing call to EVP_MD_CTX_cleanup()

FILE: ossl_pkey.c

326    EVP_VerifyUpdate(&ctx, RSTRING_PTR(data), RSTRING_LEN(data));
327    switch (EVP_VerifyFinal(&ctx, (unsigned char *)RSTRING_PTR(sig), RSTRING_LENINT(sig), pkey)) {
328    case 0:

from the openssl docs:

http://www.openssl.org/docs/crypto/EVP_VerifyInit.html

The call to EVP_VerifyFinal() internally finalizes a copy of the digest context. This means that calls to EVP_VerifyUpdate() and EVP_VerifyFinal() can be called later to digest and verify additional data.
Since only a copy of the digest context is ever finalized the context must be cleaned up after use by calling EVP_MD_CTX_cleanup() or a memory leak will occur.

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed
• % Done changed from 0 to 100

Applied in changeset r45595.

---

ossl_pkey.c: fix memory leak

• ext/openssl/ossl_pkey.c (ossl_pkey_verify): as EVP_VerifyFinal()
  finalizes only a copy of the digest context, the context must be
  cleaned up after initialization by EVP_MD_CTX_cleanup() or a
  memory leak will occur. [ruby-core:62038] [Bug #9743]

[nobu (Nobuyoshi Nakada)]

• Description updated (diff)
• Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN to 1.9.3: REQUIRED, 2.0.0: REQUIRED, 2.1: REQUIRED

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 1.9.3: REQUIRED, 2.0.0: REQUIRED, 2.1: REQUIRED to 1.9.3: REQUIRED, 2.0.0: REQUIRED, 2.1: DONE

r45595 was backported into ruby_2_1 at r45821.

[usa (Usaku NAKAMURA)]

• Backport changed from 1.9.3: REQUIRED, 2.0.0: REQUIRED, 2.1: DONE to 1.9.3: REQUIRED, 2.0.0: DONE, 2.1: DONE

backported into ruby_2_0_0 at r45868.

[vo.x (Vit Ondruch)]

This is causing test suite timeout on Fedora Rawhide ARM builder :/

https://kojipkgs.fedoraproject.org//work/tasks/4012/7074012/build.log

[vo.x (Vit Ondruch)]

• Related to Bug #9984: OpenSSL::TestPKeyRSA#test_sign_verify_memory_leak timeouts on ARM added

[zzak (zzak _)]

• Status changed from Closed to Open
• Assignee set to zzak (zzak _)
• ruby -v changed from 2.1.1 to 2.2.0

Seeing this test failure on travis:
https://travis-ci.org/zzak/openssl/jobs/48587976

I think we should re-open this ticket until its resolved.

[zzak (zzak _)]

• Status changed from Open to Closed

The failure has been fixed, so we can close this ticket.