Bug #20972: OpenSSL Memory Usage

• Status: Open
• Assignee: -
• Target version: -
• ruby -v: 3.3.6
• Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN

[mrkgrandjean (Mark Grandjean)]

While testing large file uploads I noticed the OpenSSL Buffering is allocating a lot of memory for file uploads.
Similar to the issue in this ticket from a few years ago https://bugs.ruby-lang.org/issues/14426

Using the same test code from that ticket

require "http"
require "memory_profiler"
require "stringio"

body = StringIO.new("a" * 5*1024*1024)

MemoryProfiler.report do
  HTTP.post("https://example.com", body: body)
end.pretty_print

in ruby 2.6.10

allocated memory by gem
-----------------------------------
   1913287  unicode_normalize
   1053463  llhttp-ffi-0.5.0
    187849  http-5.2.0
     50168  openssl
     36180  addressable-2.8.7
      8904  ipaddr
       272  other

While testing in Ruby 3.3.6

allocated memory by gem
-----------------------------------
  16104976  openssl
   2393456  unicode_normalize
   1053880  llhttp-ffi-0.5.0
    151633  http-5.2.0
      9813  addressable-2.8.7
      7872  ipaddr
       240  other
        40  forwardable

It looks like the memory allocation issue has returned.

[byroot (Jean Boussier)]

• Related to Feature #14426: [PATCH] openssl: reduce memory allocation in OpenSSL::Buffering#do_write added

[luke-gru (Luke Gruber)]

I took a look at this and this is due to 2 things that I could notice:

The call to string.b if it's not encoded as binary already, in the Buffer class in openssl/buffering.rb. The other one is a call to
rb_str_new_frozen in ossl_ssl_write_internal.

Getting rid of the rb_str_new_frozen call brings the allocated memory down to ~ 5639000 and removing the other one gets it down to
42840.

I'll leave it to someone with more familiarity with the openssl codebase to provide a good fix, if it's possible, possibly using locktmp or something for
the frozen case. Maybe it's also possible to just force the encoding as binary for the provided string instead of duplicating it.

[byroot (Jean Boussier)]

I think you are mostly correct, what exactly cause the memory growth is hard to point precisely (especially with memory_profiler because this codepath depends a lot on shared strings, so it's not fully clear which call exactly cause the unsharing.

But you are right that using rb_str_locktmp would avoid some allocations and probably some copying, and there are a few other small changes that can be made to rely less on string sharing, and reduce allocations a bit: https://github.com/ruby/openssl/pull/831