Bug #13846
closedOpenbsd possible memory leak when using Thread
Description
On openBSD 5.9/6.1 running this code causes RSS memory to increase without limit. This does not happen in ubuntu or other linux.
loop do
thr = Thread.new { puts "thread test" }
thr.join
GC.start
end
This is related to other things too, like when calling Open3.* , virtually everything that is using threads...
I have attached a screenshot with RSS memory plot I did.
Files
Updated by jeremyevans0 (Jeremy Evans) about 7 years ago
I've confirmed this also happens with the current version of OpenBSD and ruby 2.4.1. I'll have to write a C version to see if this is a problem with OpenBSD's thread library. If not, it's probably an issue with ruby. Hopefully I'll have time to work on that tonight.
Updated by parhs (Coding Gorilla) about 7 years ago
I installed 2.5.0-dev (from github) and seems that this bug does not occur or at least it is fixed. I believe that it might be related to #13772. My daemons started with low memory around 30mb and ended up after a week to 300mb with high cpu usage and whole system lagging. Even local ssh connection had noticeable lag.
With 2.5.0-dev that I tested RSS memory for this example seems stable. For my daemon it is increasing but very slowly compared to how it used to though it might be normal. (I got 100kb max rss increase after 6 hours compared to 4-5mb before)
Updated by kernigh (George Koehler) about 7 years ago
I can reproduce this bug in 2.3 and 2.4 but not in trunk. I am using 2.3 and 2.4 from OpenBSD packages. I compiled trunk from source.
$ dmesg | head -2
OpenBSD 6.1 (GENERIC.MP) #19: Thu Aug 3 14:59:44 CEST 2017
robert@syspatch-61-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
$ ruby -v
ruby 2.5.0dev (2017-09-06 trunk 59764) [x86_64-openbsd6.1]
$ ruby24 -v
ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-openbsd]
$ ruby23 -v
ruby 2.3.3p222 (2016-11-21 revision 56859) [x86_64-openbsd]
I run the loop in trunk ruby for some minutes, then watch the RES in top(1). It stays at 11M:
PID USERNAME PRI NICE SIZE RES STATE WAIT TIME CPU COMMAND
16837 kernigh 64 0 11M 11M onproc/0 - 6:50 94.73% ruby
When I run the loop in ruby23 or ruby24, I can see RES increasing slowly from 11M to 12M, 13M, and so on. The bug might be in Ruby 2.3 and 2.4, but not in Ruby trunk and not in OpenBSD's thread library.
Updated by jeremyevans0 (Jeremy Evans) over 5 years ago
- Status changed from Open to Closed
I did some testing of this recently. With ruby 2.6, the memory use is constant. With ruby 2.5, there is slight memory growth. However, considering the extensive thread changes between ruby 2.5 and ruby 2.6, it is unlikely the particular fix could be isolated and backported. I'm going to close this now. If you are experiencing similar memory leaks with threads with master or 2.6, please update this ticket.