Project

General

Profile

Bug #17221

Relax the Fiber#transfer's limitation

Added by ko1 (Koichi Sasada) about 2 months ago. Updated about 2 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
[ruby-core:100332]

Description

Using Fiber#transfer with Fiber#resume for a same Fiber is limited (once Fiber#transfer is called for a fiber, the fiber can not be resumed more).

require 'fiber'
f1 = nil
f2 = Fiber.new{
  f1.transfer
}
f1 = Fiber.new{
  f2.transfer
  Fiber.yield 10
  Fiber.yield 20
}
p f1.resume #=> 10
p f1.resume #=> `resume': cannot resume transferred Fiber (FiberError)

This restriction was introduced to protect the resume/yield chain, but we realized that it is too much to protect the chain.

Instead of the current restriction, we introduce some other protections.

(1) can not transfer to the resuming fiber.

require 'fiber'

root = Fiber.current
f1 = f2 = nil

f1 = Fiber.new{
  f2 = Fiber.new{
    root.transfer(10)
  }
  f2.resume
}

p f1.transfer #=> 10

# root <-----+
#  |         |
#  v         | transfer
#  f1 -> f2 -+ # resume/yield chain


# horizontal direction: resume
# vertical direction: transfer

p f1.transfer #=> attempt to transfer to a resuming fiber (FiberError)

# f1 has it's own resume/yield chain, and f1.transfer breaks the chain

# root <-----+
#  || (error)|
#  vv        |
#  f1 -> f2 -+ # resume/yield chain

(2) can not transfer to the yielding fiber.

require 'fiber'

f1 = f2 = nil

f1 = Fiber.new{
  f2 = Fiber.new{
    Fiber.yield
  }
  f2.resume
  10
}

p f1.transfer #=> 10

# root 
# | ^
# | | transfer
# v |
# f1 --> f2 # resume/yield chain
#    <--

p f2.transfer #=> `transfer': attempt to transfer to an yielding fiber (FiberError)

# f2 is waiting for the resume, so the transfer is not allowed.

# root --+
# | ^    | transfer (error)
# | |    |
# v |    v
# f1 --> f2 # resume/yield chain
#    <--

(3) can not resume transferring fiber.

require 'fiber'

f1 = f2 = nil

f2 = Fiber.new{
  f1.resume #=> attempt to resume the transferring fiber (FiberError)
}
f1 = Fiber.new{
  f2.transfer
}
f1.transfer

# root
# |
# v
# f1 <-+
# |    |
# v    | resume (error)
# f2 --+

# f1 seems waiting for transfer from other fibers.

(4) can not yield from not-resumed fiber

require 'fiber'

f2 = Fiber.new do
  Fiber.yield #=> `yield': attempt to yield on not resumed fiber (FiberError)
end

f1 = Fiber.new
  f2.transfer
end

p f1.transfer

#     root
#     |
#     v
#     f1
#     |
#     v
#  <- f2
#  yield to where ...? (2.7 switches to root fiber)

and remove current restriction. The first example works fine:

require 'fiber'
f1 = nil
f2 = Fiber.new{
  f1.transfer
}
f1 = Fiber.new{
  f2.transfer
  Fiber.yield 10
  Fiber.yield 20
}
p f1.resume #=> 10
p f1.resume #=> 20


# root -> f1 <-+
#         |    |
#         v    |
#         f2 --+

The basic idea is respect programmer's intention.

For (1), resuming fiber should be switched by the Fiber.yield.
For (2), yielding fiber should be switched by the Fiber#resume.
For (3), transferring fiber should be switched by the Fiber#transfer.

Mainly (1) can keep the resume/yield chain. Also (2) and (3) makes the chain and relationships with fibers cleanly.


Also at the end of a transferred fiber, it had continued on root fiber.

However, if the root fiber resumed a fiber (and that fiber can resumed another fiber), this behavior also breaks the resume/yield chain.
So at the end of a transferred fiber, switch to the edge of resume chain from root fiber.
For example, root fiber resumed f1 and f1 resumed f2, transferred to f3 and f3 terminated, then continue from the fiber f2 (it was continued
from root fiber without this patch).

require 'fiber'
f3 = Fiber.new{
  10
}
f2 = Fiber.new{
  f3.transfer + 20
}
f1 = Fiber.new{
  f2.resume
}
p f1.resume #=> 30

# without this patch:
#
# root -> f1 -> f2
#  ^             |
#  | exit        v
#  +----------- f3

# with this patch:
#
# root -> f1 -> f2 <-+  # keep resume/yield chain
#               |    |
#               v    |
#               f3 --+ exit

The patch is: https://github.com/ruby/ruby/pull/3636


Files

clipboard-202010080257-u2lbv.png (25.5 KB) clipboard-202010080257-u2lbv.png ko1 (Koichi Sasada), 10/07/2020 05:57 PM
#1

Updated by ko1 (Koichi Sasada) about 2 months ago

  • Description updated (diff)

Updated by ko1 (Koichi Sasada) about 2 months ago

Implementation note

Each fiber has "resuming_fiber" and we can check (1).

#3

Updated by ko1 (Koichi Sasada) about 2 months ago

  • File deleted (clipboard-202010080248-9wdyk.png)

Updated by Eregon (Benoit Daloze) about 2 months ago

This sounds great to me!

Updated by ioquatix (Samuel Williams) about 2 months ago

Great work everyone!

Updated by ko1 (Koichi Sasada) about 2 months ago

Note for (2). It can break compatibility, but transferred fibers can not be resumed more, so maybe nobody rely on mixing them.

require 'fiber'

f = Fiber.new do
  loop do
    Fiber.yield :ok
  end
end

p f.resume
p f.transfer #=> 3.0  `transfer': attempt to transfer to an yielding fiber (FiberError)
p f.resume   #=> 2.7  `resume': cannot resume transferred Fiber (FiberError)

Updated by ko1 (Koichi Sasada) about 2 months ago

  • Description updated (diff)

I found "(4) can not yield from not-resumed fiber", so I updated the proposal body with (4).

Updated by Eregon (Benoit Daloze) about 2 months ago

For (2), can we use the error message attempt to transfer to a yielding fiber (FiberError) (yielding instead of resuming)? That seems clearer since f2 is not resuming but waiting for a resume and inside Fiber.yield (= yielding).

Updated by ko1 (Koichi Sasada) about 2 months ago

  • Description updated (diff)

Eregon (Benoit Daloze) wrote in #note-8:

For (2), can we use the error message attempt to transfer to a yielding fiber (FiberError) (yielding instead of resuming)? That seems clearer since f2 is not resuming but waiting for a resume and inside Fiber.yield (= yielding).

It was my mistake. Implementation says

`transfer': attempt to transfer to an yielding fiber (FiberError)

Description is updated.

Updated by ko1 (Koichi Sasada) about 2 months ago

BTW, "a yielding"? "an yielding"?

#11

Updated by Eregon (Benoit Daloze) about 2 months ago

I believe it's a yielding Fiber (the first sound in yield-ing is not a vowel sound)

Updated by duerst (Martin Dürst) about 2 months ago

Eregon (Benoit Daloze) wrote in #note-11:

I believe it's a yielding Fiber (the first sound in yield-ing is not a vowel sound)

A Yes indeed for this question. (not an yes :-)

Updated by ko1 (Koichi Sasada) about 2 months ago

Next English question :)

"attempt to transfer to a yielding fiber" or "attempt to transfer to the yielding fiber" (a/the) on error message (error message when calling fib.transfer and fib is yielding).

Updated by ko1 (Koichi Sasada) about 2 months ago

BTW I already got Matz's approval so I'll merge it as soon as possible.

#16

Updated by ko1 (Koichi Sasada) about 2 months ago

  • Status changed from Open to Closed

Applied in changeset git|bf3b2a43741e4f72be21bc6acf24d37e7fcff61c.


relax Fiber#transfer's restriction

Using Fiber#transfer with Fiber#resume for a same Fiber is
limited (once Fiber#transfer is called for a fiber, the fiber
can not be resumed more). This restriction was introduced to
protect the resume/yield chain, but we realized that it is too much
to protect the chain. Instead of the current restriction, we
introduce some other protections.

(1) can not transfer to the resuming fiber.
(2) can not transfer to the yielding fiber.
(3) can not resume transferred fiber.
(4) can not yield from not-resumed fiber.

[Bug #17221]

Also at the end of a transferred fiber, it had continued on root fiber.
However, if the root fiber resumed a fiber (and that fiber can resumed
another fiber), this behavior also breaks the resume/yield chain.
So at the end of a transferred fiber, switch to the edge of resume
chain from root fiber.
For example, root fiber resumed f1 and f1 resumed f2, transferred to
f3 and f3 terminated, then continue from the fiber f2 (it was continued
from root fiber without this patch).

Updated by shan (Shannon Skipper) about 2 months ago

It seems like both "a" and "the" work here. I might say, "cannot transfer to a yielding Fiber" or "attempted transfer to a yielding Fiber."

Also available in: Atom PDF