Bug #1325: fiber tests kill windows

• Status: Closed
• Assignee: nobu (Nobuyoshi Nakada)
• Target version: 1.9.2
• ruby -v: ruby 1.9.2dev (2009-03-29) [i386-mswin32_90]

[cfis (Charlie Savage)]

=begin
If run on its own, test_fiber.rb runs fine on windows (VC2008 as compiler). But when run as part of the full test_suite, it eventually uses so much memory to cause windows to stop responding, start disk swapping, and finally to kill the ruby process. This consistently happens.

Reducing the number of created fibers solves the issue (patch attached), but that's surely masking some deeper issue.
=end

[rogerdpack (Roger Pack)]

=begin
I run into this too--except it just segfaults. (mingw)
=end

[yugui (Yuki Sonoda)]

• Assignee set to nobu (Nobuyoshi Nakada)
• Target version changed from 1.9.1 to 1.9.2

=begin

=end

[wanabe (_ wanabe)]

• File noalloc_fiber.patch noalloc_fiber.patch added

=begin
It's probably because the following situation.

1. fiber_init() (== Fiber#initialize) raises NoMemoryError.
2. next_fiber == NULL || next_fiber == NULL || local_storage == NULL ||
   saved_thread.stack is not allocated, but keeps original th->stack.
3. fiber_free() cause SEGV.

In this case, added Patch may prevent SEGV.
(I'm afraid that disk swapping is unavoidable.)
=end

[nobu (Nobuyoshi Nakada)]

=begin
Hi,

At Wed, 21 Oct 2009 00:45:15 +0900,
_ wanabe wrote in [ruby-core:26184]:

It's probably because the following situation.

Seems reasonable.

1. fiber_init() (== Fiber#initialize) raises NoMemoryError.
2. next_fiber == NULL || next_fiber == NULL || local_storage == NULL ||
   saved_thread.stack is not allocated, but keeps original th->stack.
3. fiber_free() cause SEGV.

In another case, when st_init_numtable() to initialize
local_storage fails, it might be left pointing the current
thread's local_storage, and cause double-free. So I suspect
fib->saved_thread.local_storage should be cleared after
cont_init() in fiber_t_alloc().

Also, setting prev_fiber and next_fiber in root_fiber_alloc()
will be no longer needed.

--
Nobu Nakada

=end

[wanabe (_ wanabe)]

=begin
Thank you, Nobu. I rewrote the patch in accordance with your advise.

And finally, I found which makes disk-swapping.
File.open('large.png', 'r:ascii-8bit') in test/cgi/test_cgi_multipart.rb does.
It does realloc many times and increases malloc_limit in gc.c.
Interestingly, mode 'rb:ascii-8bit' gets rid of the problem.

This is another example that enlargement of malloc_limit obstructs GC.

$ cat test.rb
require 'benchmark'
a = []
10.times do |i|
ARGV[0].to_i.times do
a[10000] = nil
a.clear
end
end

GC.start
GC::Profiler.enable

Benchmark.bm do |bm|
bm.report do
100_000.times do
Fiber.new{}.resume
end
end
end

gc_result = GC::Profiler.result.split(/\n/)
gc_result[4..-3] = "(snip)"
puts nil, gc_result

$ ./ruby -v test.rb 0
ruby 1.9.2dev (2009-10-30 trunk 25566) [i386-mingw32]
user system total real
4.516000 1.844000 6.360000 ( 7.109375)

GC 294 invokes.
Index Invoke Time(sec) Use Size(byte) Total Size(byte) Total Object GC Time(ms)
1 0.109 70704 229376 9548 0.00000000000000000000
2 0.109 70776 229376 9548 15.62500000000000000000
(snip)
292 4.594 70776 229376 9548 0.00000000000000000000
293 4.609 70776 229376 9548 0.00000000000000000000

$ ./ruby -v test.rb 10000
ruby 1.9.2dev (2009-10-30 trunk 25566) [i386-mingw32]
user system total real
6.406000 3.406000 9.812000 (108.718750)

GC 6 invokes.
Index Invoke Time(sec) Use Size(byte) Total Size(byte) Total Object GC Time(ms)
1 5.375 70728 245760 10230 0.00000000000000000000
2 5.562 70752 425984 17732 0.00000000000000000000
(snip)
4 6.844 70848 1343488 55924 0.00000000000000000000
5 9.266 70872 2392064 99572 0.00000000000000000000
=end

[wanabe (_ wanabe)]

• Status changed from Open to Closed

=begin
Fiber tests doesn't kill windows now.
So, I close this issue. Thanks.
=end