Project

General

Profile

Feature #15144

Enumerator#chain

Added by zverok (Victor Shepelev) about 1 year ago. Updated 12 months ago.

Status:
Closed
Priority:
Normal
Target version:
-
[ruby-core:89121]

Description

I am not sure I am not missing something, but...

[1, 2, 3].each.chain([3, 4, 5].each) # => Enumerator

...seem to be a useful pattern.

It especially shows itself in case of lazy enumerators, representing several long-calculated sequences, like something...

# just data from several sources, abstracted into enumerator, fetching it on demand
process = URLS.lazy.map(&Faraday.method(:get)))
  .chain(LOCAL_FILES.lazy.map(&File.method(:read)))
  .chain(FALLBACK_FILE.then.lazy.map(&File.method(:read))) # with yield_self aka then we can even chain ONE value

process.detect { |val| found?(val) } # uniformely search several sources (lazy-loading them) for some value

# tty-progressbar is able to work with enumerables:
bar = TTY::ProgressBar.new("[:bar]", total: URLS.count + LOCAL_FILES.count + 1)
bar.iterate(process).detect { |val| found?(val) } # shows progress-bar for uniform process of detection

Prototype impl. is dead simple, of course:

class Enumerator
  def chain(*others)
    Enumerator.new { |y|
      [self, *others].each { |e| e.each { |v| y << v } }
    }
  end
end

Obviously, the effect could be reached with flat_map, but it seems "chaining" of iterations is pretty common and clear concept (and Google search for "ruby enumerator chain" shows people constantly ask about the way).


Files


Related issues

Related to Ruby master - Feature #709: Enumerator#+RejectedActions

Associated revisions

Revision 045b0e54
Added by knu (Akinori MUSHA) 12 months ago

Implement Enumerator#+ and Enumerable#chain [Feature #15144]

They return an Enumerator::Chain object which is a subclass of
Enumerator, which represents a chain of enumerables that works as a
single enumerator.

e = (1..3).chain([4, 5])
e.to_a #=> [1, 2, 3, 4, 5]

e = (1..3).each + [4, 5]
e.to_a #=> [1, 2, 3, 4, 5]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65949 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 65949
Added by knu (Akinori MUSHA) 12 months ago

Implement Enumerator#+ and Enumerable#chain [Feature #15144]

They return an Enumerator::Chain object which is a subclass of
Enumerator, which represents a chain of enumerables that works as a
single enumerator.

e = (1..3).chain([4, 5])
e.to_a #=> [1, 2, 3, 4, 5]

e = (1..3).each + [4, 5]
e.to_a #=> [1, 2, 3, 4, 5]

Revision 65949
Added by knu (Akinori MUSHA) 12 months ago

Implement Enumerator#+ and Enumerable#chain [Feature #15144]

They return an Enumerator::Chain object which is a subclass of
Enumerator, which represents a chain of enumerables that works as a
single enumerator.

e = (1..3).chain([4, 5])
e.to_a #=> [1, 2, 3, 4, 5]

e = (1..3).each + [4, 5]
e.to_a #=> [1, 2, 3, 4, 5]

History

Updated by shevegen (Robert A. Heiler) about 1 year ago

Obviously, the effect could be reached with flat_map

I always found that name weird so ... assumingly that it is the same
as .flatten.map, I always used the latter. :-)

(I don't recall flat_map offhand and I happily admit that I have not
googled; I try to keep ruby so simple to need only few things if
possible.)

but it seems "chaining" of iterations is pretty common and clear
concept (and Google search for "ruby enumerator chain" shows
people constantly ask about the way).

Well, I think this may have more to do how to name something. It may
be best to actually ask matz about the "chaining" here.

My personal opinion, which may be wrong, is that the term chaining
is used here just to put method after method onto an object (e. g.
send message after message to your object) - and have it perform
the corresponding code defined in these methods. A bit like a stream
of data through a pipe, a filter.

If this is a correct point of view (and I really don't know; we may
have to ask matz), then there should not be a need to call it
a "chain" - but most definitely to not use it as a word for a new
or additional method, be it .chain() or .chaining(). But again,
ultimately only matz knows.

By the way:

[1, 2, 3].each.chain([3, 4, 5].each) 

The repetition of .each seems a bit awkward and the intention is
also not very clear to me. But that is just my personal opinion;
people write ruby code in very different ways. The above code
looks alien to me too. :)

#2

Updated by knu (Akinori MUSHA) about 1 year ago

Updated by knu (Akinori MUSHA) about 1 year ago

In today's developer meeting, Matz said Enumerator#+ would be OK to add, so I'm going to work on it first and then we'll think about an alias, and a constructor that takes many enumerators later.

Updated by knu (Akinori MUSHA) 12 months ago

  • Assignee set to matz (Yukihiro Matsumoto)
  • Status changed from Open to Assigned

I'm working on this and the implementation of Enumerator#+(enum) and Enumerator#chain(*enums) is about to complete.

Matz, what do you think about the name "chain"? Python has chain(), Rust too has chain, and I cannot think of any better name.

Updated by knu (Akinori MUSHA) 12 months ago

I've written an initial implementation as attached:

  • Enumerator.chain(*enums) to generate an enumerator chain of enums
  • Enumerator#+(other) to generate an enumerator chain of [self, other]
  • Enumerator#chain(*others) to generate an enumerator chain of [self, *others]

Some notes:

  • The constructor is currently Enumerator::Chain.new(*enums) but it should probably be Enumerator::Chain.new(enums) to make it extensible to take an enumerable in general, with itertools.chain.from_iterable of Python in mind.
  • Enumerator.chain(Enumerator.chain(e1, e2), e3) cannot be optimized to Enumerator.chain(e1, e2, e3) because it is expected that the intermediate object Enumerator.chain(e1, e2) receive a call for each when Enumerator.chain(Enumerator.chain(e1, e2), e3).each {…} is called.

Updated by knu (Akinori MUSHA) 12 months ago

knu (Akinori MUSHA) wrote:

  • The constructor is currently Enumerator::Chain.new(*enums) but it should probably be Enumerator::Chain.new(enums) to make it extensible to take an enumerable in general, with itertools.chain.from_iterable of Python in mind.

This could be implemented as a different class if needed, because it would have little in common with an array-based chain.

Updated by knu (Akinori MUSHA) 12 months ago

We got Matz's approval for adding Enumerable#chain (instead of Enumerator#chain) and Enumerator#+.

Updated by knu (Akinori MUSHA) 12 months ago

  • Assignee changed from matz (Yukihiro Matsumoto) to knu (Akinori MUSHA)
#9

Updated by knu (Akinori MUSHA) 12 months ago

  • Status changed from Assigned to Closed

Applied in changeset trunk|r65949.


Implement Enumerator#+ and Enumerable#chain [Feature #15144]

They return an Enumerator::Chain object which is a subclass of
Enumerator, which represents a chain of enumerables that works as a
single enumerator.

e = (1..3).chain([4, 5])
e.to_a #=> [1, 2, 3, 4, 5]

e = (1..3).each + [4, 5]
e.to_a #=> [1, 2, 3, 4, 5]

Also available in: Atom PDF