Project

General

Profile

Actions

Bug #13632

closed

Not processable interrupt queue for a thread after it's notified that FD is closed in some other thread.

Added by nvashchenko (Nikolay Vashchenko) over 7 years ago. Updated over 7 years ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
2.2.x, 2.3.x, 2.4.x, 2.5.x
[ruby-core:81581]

Description

In the bugfix for https://bugs.ruby-lang.org/issues/13076 has been introduced another bug, caused by a busy waiting in rb_notify_fd_close method, while the FD is being released. During this waiting, it pumps huge amounts of the ruby_error_stream_closed errors into thread's interrupt queue, which almost all stay there unprocessed. It can be up for several hundred of exceptions in the queue, depending on circumstances.

  a = []
  t = []
  10.times do
    r,w = IO.pipe
    a << [r,w]
    t << Thread.new do
      while r.gets
      end rescue IOError
      # Interrupt queue is full and all IO is broken
     Thread.current.pending_interrupt? # Expected to be false, because it's already rescued
      IO.sysopen ("/dev/tty")                   # Expected not to throw an error. On each such call, it dequeues the next item from interrupt queue until there's none
    end
  end
  a.each do |r,w|
    w.puts "test"
    w.close
    r.close
  end
  t.each do |th|
    th.join
  end

Output:

Traceback (most recent call last):
	1: from test2.rb:9:in `block (2 levels) in <main>'
test2.rb:9:in `sysopen': stream closed in another thread (IOError)
Actions #1

Updated by nvashchenko (Nikolay Vashchenko) over 7 years ago

  • Description updated (diff)
Actions #2

Updated by nvashchenko (Nikolay Vashchenko) over 7 years ago

  • Description updated (diff)

Updated by normalperson (Eric Wong) over 7 years ago

wrote:

Bug #13632: Not processable interrupt queue for a thread after it's notified that FD is closed in some other thread.
https://bugs.ruby-lang.org/issues/13632

Investigating. I've been looking at IO close notification for
[Feature #13618] (auto-Fiber) anyways.

Actions #4

Updated by Anonymous over 7 years ago

  • Status changed from Open to Closed

Applied in changeset trunk|r59020.


IO#close: do not enqueue redundant interrupts

Enqueuing multiple errors for one event causes spurious errors
down the line, as reported by Nikolay Vashchenko in
https://bugs.ruby-lang.org/issues/13632

  • thread.c (rb_notify_fd_close): do not enqueue multiple interrupts
    [ruby-core:81581] [Bug #13632]
  • test/ruby/test_io.rb (test_single_exception_on_close):
    new test based on script from Nikolay

Updated by normalperson (Eric Wong) over 7 years ago

wrote:

https://bugs.ruby-lang.org/issues/13632

r59020 should fix it trivially in trunk.

Backporting to <= 2.4 is only a little different due to the
data structure change:

s/wfd->fd = -1/th->waiting_fd = -1/
https://80x24.org/spew/20170606001646.22889-1-e@80x24.org/raw

Thanks for the report!

Actions #6

Updated by ko1 (Koichi Sasada) over 7 years ago

  • Description updated (diff)

Updated by normalperson (Eric Wong) over 7 years ago

Eric Wong wrote:

wrote:

https://bugs.ruby-lang.org/issues/13632

r59020 should fix it trivially in trunk.

Make that r59028 :x r59020 interacted badly with r57422

Backporting to <= 2.4 is only a little different due to the
data structure change:

r59028 backporting is more difficult.

Below are links to backported patches from trunk to fix [Bug #13632] for
Ruby 2.4 maintenance branches and earlier. I mainly wanted to
backport r59028, but that depends on r58812 (originally intended
as a pure performance optimization).

I'd rather not change r59028 to something unrecognizable from
what is in trunk for backporting, as that might negatively
impact future backports.

r58812: speed up IO#close with many threads
https://80x24.org/spew/20170607195901.18958-2-e@80x24.org/raw
r59028: IO#close: do not enqueue redundant interrupts (take #2)
https://80x24.org/spew/20170607195901.18958-3-e@80x24.org/raw

test/ruby/test_io.rb | 22 ++++++++++++++++++++++
thread.c | 33 ++++++++++++++++++++++++---------
vm.c | 1 -
vm_core.h | 4 ++--
4 files changed, 48 insertions(+), 12 deletions(-)

Actions #8

Updated by usa (Usaku NAKAMURA) over 7 years ago

  • Backport changed from 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN to 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: REQUIRED
Actions #9

Updated by usa (Usaku NAKAMURA) over 7 years ago

  • Backport changed from 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: REQUIRED to 2.2: UNKNOWN, 2.3: DONE, 2.4: REQUIRED

Updated by nagachika (Tomoyuki Chikanaga) over 7 years ago

  • Backport changed from 2.2: UNKNOWN, 2.3: DONE, 2.4: REQUIRED to 2.2: UNKNOWN, 2.3: DONE, 2.4: DONE

ruby_2_4 r59286 merged revision(s) 58284,58812,59028.

Updated by nvashchenko (Nikolay Vashchenko) over 7 years ago

I created a gem with a temporary workaround for versions 2.2.7, 2.3.4 and 2.4.1 to help folks until versions with backports are released:
https://rubygems.org/gems/stopgap_13632

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0