Bug #10735: Memory leak in openssl ossl_pkey_sign

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]
• Backport: 2.0.0: REQUIRED, 2.1: DONE, 2.2: DONE

[viktor (Viktor Vasilev)]

Similar to the memory leak fixed in https://bugs.ruby-lang.org/issues/9743 there is an issue with ossl_pkey_sign. The ruby heap usage reported through GC.stat remains very stable, while the process heap grows linearly with the number of OpenSSL::PKey::RSA sign calls.
The documentation at https://www.openssl.org/docs/crypto/EVP_SignInit.html (similar to EVP_VerifyInit) mentions that not disposing the context causes a leak.

To reproduce: https://gist.github.com/viktorium/f032cdc8906f43dac94e
A patch with a fix very similar to issue #9743: https://gist.github.com/viktorium/b466b72c83d2ab90182c

[zzak (zzak _)]

• Status changed from Open to Assigned
• Assignee set to zzak (zzak _)

I know you tried this with 1.9.3, but could you try to repro on trunk (and newer rubies) first?

1.9.3 will be EOL soon, and I want to make sure we fix it upstream before attempting any backports

[zzak (zzak _)]

• Assignee changed from zzak (zzak _) to 7150
• Priority changed from 5 to Normal

[viktor (Viktor Vasilev)]

Zachary Scott wrote:

I know you tried this with 1.9.3, but could you try to repro on trunk (and newer rubies) first?

1.9.3 will be EOL soon, and I want to make sure we fix it upstream before attempting any backports

Hi Zachary,

Just ran the test case against Ruby 2.3.0dev (2015-01-12 trunk 49226) [x86_64-darwin14] and see the exact same memory leak:

{:count=>7, :heap_allocated_pages=>74, :heap_sorted_length=>75, :heap_allocatable_pages=>0, :heap_available_slots=>30161, :heap_live_slots=>29720, :heap_free_slots=>441, :heap_final_slots=>0, :heap_marked_slots=>11592, :heap_swept_slots=>10966, :heap_eden_pages=>74, :heap_tomb_pages=>0, :total_allocated_pages=>74, :total_freed_pages=>0, :total_allocated_objects=>91749, :total_freed_objects=>62029, :malloc_increase_bytes=>530256, :malloc_increase_bytes_limit=>16777216, :minor_gc_count=>5, :major_gc_count=>2, :remembered_wb_unprotected_objects=>180, :remembered_wb_unprotected_objects_limit=>278, :old_objects=>10540, :old_objects_limit=>10818, :oldmalloc_increase_bytes=>1808128, :oldmalloc_increase_bytes_limit=>16777216}
Memory 11736KB

<< 100_000 iterations of RSA sign >>

{:count=>25, :heap_allocated_pages=>74, :heap_sorted_length=>75, :heap_allocatable_pages=>0, :heap_available_slots=>30161, :heap_live_slots=>30108, :heap_free_slots=>53, :heap_final_slots=>0, :heap_marked_slots=>13570, :heap_swept_slots=>11362, :heap_eden_pages=>74, :heap_tomb_pages=>0, :total_allocated_pages=>74, :total_freed_pages=>0, :total_allocated_objects=>392910, :total_freed_objects=>362802, :malloc_increase_bytes=>15616, :malloc_increase_bytes_limit=>16777216, :minor_gc_count=>22, :major_gc_count=>3, :remembered_wb_unprotected_objects=>298, :remembered_wb_unprotected_objects_limit=>596, :old_objects=>13151, :old_objects_limit=>26046, :oldmalloc_increase_bytes=>39904, :oldmalloc_increase_bytes_limit=>16777216}
Memory 26244KB

Let me know if I can provide further information.

[tonci (Tonči Damjanić)]

Confirming the same with the current Ruby 2.2 (ruby 2.2.0p0 (2014-12-25 revision 49005) [x86_64-darwin14]):

{:count=>5, :heap_allocated_pages=>74, :heap_sorted_length=>75, :heap_allocatable_pages=>0, :heap_available_slots=>30164, :heap_live_slots=>29039, :heap_free_slots=>1125, :heap_final_slots=>0, :heap_marked_slots=>8424, :heap_swept_slots=>9725, :heap_eden_pages=>73, :heap_tomb_pages=>1, :total_allocated_pages=>74, :total_freed_pages=>0, :total_allocated_objects=>52920, :total_freed_objects=>23881, :malloc_increase_bytes=>275968, :malloc_increase_bytes_limit=>16777216, :minor_gc_count=>3, :major_gc_count=>2, :remembered_wb_unprotected_objects=>161, :remembered_wb_unprotected_objects_limit=>278, :old_objects=>8196, :old_objects_limit=>10808, :oldmalloc_increase_bytes=>276352, :oldmalloc_increase_bytes_limit=>16777216}
Memory 10188KB

100k iterations later:

{:count=>20, :heap_allocated_pages=>74, :heap_sorted_length=>75, :heap_allocatable_pages=>0, :heap_available_slots=>30164, :heap_live_slots=>29999, :heap_free_slots=>165, :heap_final_slots=>0, :heap_marked_slots=>9656, :heap_swept_slots=>406, :heap_eden_pages=>74, :heap_tomb_pages=>0, :total_allocated_pages=>74, :total_freed_pages=>0, :total_allocated_objects=>353037, :total_freed_objects=>323038, :malloc_increase_bytes=>24000, :malloc_increase_bytes_limit=>16777216, :minor_gc_count=>18, :major_gc_count=>2, :remembered_wb_unprotected_objects=>227, :remembered_wb_unprotected_objects_limit=>278, :old_objects=>9331, :old_objects_limit=>10808, :oldmalloc_increase_bytes=>1983872, :oldmalloc_increase_bytes_limit=>16777216}
Memory 25340KB

[zzak (zzak _)]

I've applied the patch to a branch, if you're on Ruby 2.2 you can try it out by adding this to your Gemfile:

gem "openssl", github: "ruby/openssl", branch: "ruby-bug-10735"

[zzak (zzak _)]

• Status changed from Assigned to Closed

Applied in changeset r52556.

---

• ext/openssl/ossl_pkey.c: Merge ruby/openssl@b9ea8ef [Bug #10735]

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN to 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: REQUIRED

[usa (Usaku NAKAMURA)]

• Backport changed from 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: REQUIRED to 2.0.0: REQUIRED, 2.1: DONE, 2.2: REQUIRED

ruby_2_1 r52643 merged revision(s) 52556,52557.

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.0.0: REQUIRED, 2.1: DONE, 2.2: REQUIRED to 2.0.0: REQUIRED, 2.1: DONE, 2.2: DONE

Backported into ruby_2_2 branch at r52651.