Project

General

Profile

Bug #1325

fiber tests kill windows

Added by cfis (Charlie Savage) about 10 years ago. Updated almost 8 years ago.

Status:
Closed
Priority:
Normal
Target version:
ruby -v:
ruby 1.9.2dev (2009-03-29) [i386-mswin32_90]
Backport:
[ruby-core:23053]

Description

=begin
If run on its own, test_fiber.rb runs fine on windows (VC2008 as compiler). But when run as part of the full test_suite, it eventually uses so much memory to cause windows to stop responding, start disk swapping, and finally to kill the ruby process. This consistently happens.

Reducing the number of created fibers solves the issue (patch attached), but that's surely masking some deeper issue.
=end


Files

test_fiber.patch (489 Bytes) test_fiber.patch cfis (Charlie Savage), 03/29/2009 04:27 PM
noalloc_fiber.patch (1.01 KB) noalloc_fiber.patch wanabe (_ wanabe), 10/21/2009 12:45 AM

Associated revisions

Revision b23461b5
Added by wanabe (_ wanabe) over 9 years ago

  • cont.c (fiber_free): don't free unallocated local_storage. see #1325.

  • cont.c (cont_init): clear local_storage not to use current thread's.

  • cont.c (fiber_t_alloc, root_fiber_alloc): link itself always for
    a case that fiber_link_remove() is called before fiber_link_join().

  • cont.c (fiber_init): don't join half-baked fiber.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25623 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 25623
Added by wanabe (_ wanabe) over 9 years ago

  • cont.c (fiber_free): don't free unallocated local_storage. see #1325.

  • cont.c (cont_init): clear local_storage not to use current thread's.

  • cont.c (fiber_t_alloc, root_fiber_alloc): link itself always for
    a case that fiber_link_remove() is called before fiber_link_join().

  • cont.c (fiber_init): don't join half-baked fiber.

Revision 25623
Added by wanabe (_ wanabe) over 9 years ago

  • cont.c (fiber_free): don't free unallocated local_storage. see #1325.

  • cont.c (cont_init): clear local_storage not to use current thread's.

  • cont.c (fiber_t_alloc, root_fiber_alloc): link itself always for
    a case that fiber_link_remove() is called before fiber_link_join().

  • cont.c (fiber_init): don't join half-baked fiber.

Revision 25623
Added by wanabe (_ wanabe) over 9 years ago

  • cont.c (fiber_free): don't free unallocated local_storage. see #1325.

  • cont.c (cont_init): clear local_storage not to use current thread's.

  • cont.c (fiber_t_alloc, root_fiber_alloc): link itself always for
    a case that fiber_link_remove() is called before fiber_link_join().

  • cont.c (fiber_init): don't join half-baked fiber.

Revision 25623
Added by wanabe (_ wanabe) over 9 years ago

  • cont.c (fiber_free): don't free unallocated local_storage. see #1325.

  • cont.c (cont_init): clear local_storage not to use current thread's.

  • cont.c (fiber_t_alloc, root_fiber_alloc): link itself always for
    a case that fiber_link_remove() is called before fiber_link_join().

  • cont.c (fiber_init): don't join half-baked fiber.

Revision 25623
Added by wanabe (_ wanabe) over 9 years ago

  • cont.c (fiber_free): don't free unallocated local_storage. see #1325.

  • cont.c (cont_init): clear local_storage not to use current thread's.

  • cont.c (fiber_t_alloc, root_fiber_alloc): link itself always for
    a case that fiber_link_remove() is called before fiber_link_join().

  • cont.c (fiber_init): don't join half-baked fiber.

Revision 25623
Added by wanabe (_ wanabe) over 9 years ago

  • cont.c (fiber_free): don't free unallocated local_storage. see #1325.

  • cont.c (cont_init): clear local_storage not to use current thread's.

  • cont.c (fiber_t_alloc, root_fiber_alloc): link itself always for
    a case that fiber_link_remove() is called before fiber_link_join().

  • cont.c (fiber_init): don't join half-baked fiber.

Revision 2c2fb9c5
Added by wanabe (_ wanabe) over 9 years ago

  • cont.c (fiber_free): don't free unallocated local_storage. see #1325.

  • cont.c (cont_init): clear local_storage not to use current thread's.

  • cont.c (fiber_t_alloc, root_fiber_alloc): link itself always for
    a case that fiber_link_remove() is called before fiber_link_join().

  • cont.c (fiber_init): clear cont->vm_stack and th->stack before
    root_fiber_alloc() in rb_fiber_current().

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25631 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 25631
Added by wanabe (_ wanabe) over 9 years ago

  • cont.c (fiber_free): don't free unallocated local_storage. see #1325.

  • cont.c (cont_init): clear local_storage not to use current thread's.

  • cont.c (fiber_t_alloc, root_fiber_alloc): link itself always for
    a case that fiber_link_remove() is called before fiber_link_join().

  • cont.c (fiber_init): clear cont->vm_stack and th->stack before
    root_fiber_alloc() in rb_fiber_current().

Revision 25631
Added by wanabe (_ wanabe) over 9 years ago

  • cont.c (fiber_free): don't free unallocated local_storage. see #1325.

  • cont.c (cont_init): clear local_storage not to use current thread's.

  • cont.c (fiber_t_alloc, root_fiber_alloc): link itself always for
    a case that fiber_link_remove() is called before fiber_link_join().

  • cont.c (fiber_init): clear cont->vm_stack and th->stack before
    root_fiber_alloc() in rb_fiber_current().

Revision 25631
Added by wanabe (_ wanabe) over 9 years ago

  • cont.c (fiber_free): don't free unallocated local_storage. see #1325.

  • cont.c (cont_init): clear local_storage not to use current thread's.

  • cont.c (fiber_t_alloc, root_fiber_alloc): link itself always for
    a case that fiber_link_remove() is called before fiber_link_join().

  • cont.c (fiber_init): clear cont->vm_stack and th->stack before
    root_fiber_alloc() in rb_fiber_current().

Revision 25631
Added by wanabe (_ wanabe) over 9 years ago

  • cont.c (fiber_free): don't free unallocated local_storage. see #1325.

  • cont.c (cont_init): clear local_storage not to use current thread's.

  • cont.c (fiber_t_alloc, root_fiber_alloc): link itself always for
    a case that fiber_link_remove() is called before fiber_link_join().

  • cont.c (fiber_init): clear cont->vm_stack and th->stack before
    root_fiber_alloc() in rb_fiber_current().

Revision 25631
Added by wanabe (_ wanabe) over 9 years ago

  • cont.c (fiber_free): don't free unallocated local_storage. see #1325.

  • cont.c (cont_init): clear local_storage not to use current thread's.

  • cont.c (fiber_t_alloc, root_fiber_alloc): link itself always for
    a case that fiber_link_remove() is called before fiber_link_join().

  • cont.c (fiber_init): clear cont->vm_stack and th->stack before
    root_fiber_alloc() in rb_fiber_current().

Revision 25631
Added by wanabe (_ wanabe) over 9 years ago

  • cont.c (fiber_free): don't free unallocated local_storage. see #1325.

  • cont.c (cont_init): clear local_storage not to use current thread's.

  • cont.c (fiber_t_alloc, root_fiber_alloc): link itself always for
    a case that fiber_link_remove() is called before fiber_link_join().

  • cont.c (fiber_init): clear cont->vm_stack and th->stack before
    root_fiber_alloc() in rb_fiber_current().

History

#1

Updated by rogerdpack (Roger Pack) almost 10 years ago

=begin
I run into this too--except it just segfaults. (mingw)
=end

#2

Updated by yugui (Yuki Sonoda) almost 10 years ago

  • Assignee set to nobu (Nobuyoshi Nakada)
  • Target version changed from 1.9.1 to 1.9.2

=begin

=end

#3

Updated by wanabe (_ wanabe) over 9 years ago

=begin
It's probably because the following situation.

  1. fiber_init() (== Fiber#initialize) raises NoMemoryError.
  2. next_fiber == NULL || next_fiber == NULL || local_storage == NULL || saved_thread.stack is not allocated, but keeps original th->stack.
  3. fiber_free() cause SEGV.

In this case, added Patch may prevent SEGV.
(I'm afraid that disk swapping is unavoidable.)
=end

#4

Updated by nobu (Nobuyoshi Nakada) over 9 years ago

=begin
Hi,

At Wed, 21 Oct 2009 00:45:15 +0900,
_ wanabe wrote in [ruby-core:26184]:

It's probably because the following situation.

Seems reasonable.

  1. fiber_init() (== Fiber#initialize) raises NoMemoryError.
  2. next_fiber == NULL || next_fiber == NULL || local_storage == NULL || saved_thread.stack is not allocated, but keeps original th->stack.
  3. fiber_free() cause SEGV.

In another case, when st_init_numtable() to initialize
local_storage fails, it might be left pointing the current
thread's local_storage, and cause double-free. So I suspect
fib->saved_thread.local_storage should be cleared after
cont_init() in fiber_t_alloc().

Also, setting prev_fiber and next_fiber in root_fiber_alloc()
will be no longer needed.

--
Nobu Nakada

=end

#5

Updated by wanabe (_ wanabe) over 9 years ago

=begin
Thank you, Nobu. I rewrote the patch in accordance with your advise.

And finally, I found which makes disk-swapping.
File.open('large.png', 'r:ascii-8bit') in test/cgi/test_cgi_multipart.rb does.
It does realloc many times and increases malloc_limit in gc.c.
Interestingly, mode 'rb:ascii-8bit' gets rid of the problem.

This is another example that enlargement of malloc_limit obstructs GC.

$ cat test.rb
require 'benchmark'
a = []
10.times do |i|
ARGV[0].to_i.times do
a[10000] = nil
a.clear
end
end

GC.start
GC::Profiler.enable

Benchmark.bm do |bm|
bm.report do
100_000.times do
Fiber.new{}.resume
end
end
end

gc_result = GC::Profiler.result.split(/\n/)
gc_result[4..-3] = "(snip)"
puts nil, gc_result

$ ./ruby -v test.rb 0
ruby 1.9.2dev (2009-10-30 trunk 25566) [i386-mingw32]
user system total real
4.516000 1.844000 6.360000 ( 7.109375)

GC 294 invokes.
Index Invoke Time(sec) Use Size(byte) Total Size(byte) Total Object GC Time(ms)
1 0.109 70704 229376 9548 0.00000000000000000000
2 0.109 70776 229376 9548 15.62500000000000000000
(snip)
292 4.594 70776 229376 9548 0.00000000000000000000
293 4.609 70776 229376 9548 0.00000000000000000000

$ ./ruby -v test.rb 10000
ruby 1.9.2dev (2009-10-30 trunk 25566) [i386-mingw32]
user system total real
6.406000 3.406000 9.812000 (108.718750)

GC 6 invokes.
Index Invoke Time(sec) Use Size(byte) Total Size(byte) Total Object GC Time(ms)
1 5.375 70728 245760 10230 0.00000000000000000000
2 5.562 70752 425984 17732 0.00000000000000000000
(snip)
4 6.844 70848 1343488 55924 0.00000000000000000000
5 9.266 70872 2392064 99572 0.00000000000000000000
=end

#6

Updated by wanabe (_ wanabe) almost 9 years ago

  • Status changed from Open to Closed

=begin
Fiber tests doesn't kill windows now.
So, I close this issue. Thanks.
=end

Also available in: Atom PDF