Project

General

Profile

Actions

Feature #15144

closed

Enumerator#chain

Added by zverok (Victor Shepelev) over 5 years ago. Updated over 5 years ago.

Status:
Closed
Target version:
-
[ruby-core:89121]

Description

I am not sure I am not missing something, but...

[1, 2, 3].each.chain([3, 4, 5].each) # => Enumerator

...seem to be a useful pattern.

It especially shows itself in case of lazy enumerators, representing several long-calculated sequences, like something...

# just data from several sources, abstracted into enumerator, fetching it on demand
process = URLS.lazy.map(&Faraday.method(:get)))
  .chain(LOCAL_FILES.lazy.map(&File.method(:read)))
  .chain(FALLBACK_FILE.then.lazy.map(&File.method(:read))) # with yield_self aka then we can even chain ONE value

process.detect { |val| found?(val) } # uniformely search several sources (lazy-loading them) for some value

# tty-progressbar is able to work with enumerables:
bar = TTY::ProgressBar.new("[:bar]", total: URLS.count + LOCAL_FILES.count + 1)
bar.iterate(process).detect { |val| found?(val) } # shows progress-bar for uniform process of detection

Prototype impl. is dead simple, of course:

class Enumerator
  def chain(*others)
    Enumerator.new { |y|
      [self, *others].each { |e| e.each { |v| y << v } }
    }
  end
end

Obviously, the effect could be reached with flat_map, but it seems "chaining" of iterations is pretty common and clear concept (and Google search for "ruby enumerator chain" shows people constantly ask about the way).


Files


Related issues 1 (0 open1 closed)

Related to Ruby master - Feature #709: Enumerator#+Rejectedknu (Akinori MUSHA)Actions

Updated by shevegen (Robert A. Heiler) over 5 years ago

Obviously, the effect could be reached with flat_map

I always found that name weird so ... assumingly that it is the same
as .flatten.map, I always used the latter. :-)

(I don't recall flat_map offhand and I happily admit that I have not
googled; I try to keep ruby so simple to need only few things if
possible.)

but it seems "chaining" of iterations is pretty common and clear
concept (and Google search for "ruby enumerator chain" shows
people constantly ask about the way).

Well, I think this may have more to do how to name something. It may
be best to actually ask matz about the "chaining" here.

My personal opinion, which may be wrong, is that the term chaining
is used here just to put method after method onto an object (e. g.
send message after message to your object) - and have it perform
the corresponding code defined in these methods. A bit like a stream
of data through a pipe, a filter.

If this is a correct point of view (and I really don't know; we may
have to ask matz), then there should not be a need to call it
a "chain" - but most definitely to not use it as a word for a new
or additional method, be it .chain() or .chaining(). But again,
ultimately only matz knows.

By the way:

[1, 2, 3].each.chain([3, 4, 5].each) 

The repetition of .each seems a bit awkward and the intention is
also not very clear to me. But that is just my personal opinion;
people write ruby code in very different ways. The above code
looks alien to me too. :)

Actions #2

Updated by knu (Akinori MUSHA) over 5 years ago

Updated by knu (Akinori MUSHA) over 5 years ago

In today's developer meeting, Matz said Enumerator#+ would be OK to add, so I'm going to work on it first and then we'll think about an alias, and a constructor that takes many enumerators later.

Updated by knu (Akinori MUSHA) over 5 years ago

  • Status changed from Open to Assigned
  • Assignee set to matz (Yukihiro Matsumoto)

I'm working on this and the implementation of Enumerator#+(enum) and Enumerator#chain(*enums) is about to complete.

Matz, what do you think about the name "chain"? Python has chain(), Rust too has chain, and I cannot think of any better name.

Updated by knu (Akinori MUSHA) over 5 years ago

I've written an initial implementation as attached:

  • Enumerator.chain(*enums) to generate an enumerator chain of enums
  • Enumerator#+(other) to generate an enumerator chain of [self, other]
  • Enumerator#chain(*others) to generate an enumerator chain of [self, *others]

Some notes:

  • The constructor is currently Enumerator::Chain.new(*enums) but it should probably be Enumerator::Chain.new(enums) to make it extensible to take an enumerable in general, with itertools.chain.from_iterable of Python in mind.
  • Enumerator.chain(Enumerator.chain(e1, e2), e3) cannot be optimized to Enumerator.chain(e1, e2, e3) because it is expected that the intermediate object Enumerator.chain(e1, e2) receive a call for each when Enumerator.chain(Enumerator.chain(e1, e2), e3).each {…} is called.

Updated by knu (Akinori MUSHA) over 5 years ago

knu (Akinori MUSHA) wrote:

  • The constructor is currently Enumerator::Chain.new(*enums) but it should probably be Enumerator::Chain.new(enums) to make it extensible to take an enumerable in general, with itertools.chain.from_iterable of Python in mind.

This could be implemented as a different class if needed, because it would have little in common with an array-based chain.

Updated by knu (Akinori MUSHA) over 5 years ago

We got Matz's approval for adding Enumerable#chain (instead of Enumerator#chain) and Enumerator#+.

Updated by knu (Akinori MUSHA) over 5 years ago

  • Assignee changed from matz (Yukihiro Matsumoto) to knu (Akinori MUSHA)
Actions #9

Updated by knu (Akinori MUSHA) over 5 years ago

  • Status changed from Assigned to Closed

Applied in changeset trunk|r65949.


Implement Enumerator#+ and Enumerable#chain [Feature #15144]

They return an Enumerator::Chain object which is a subclass of
Enumerator, which represents a chain of enumerables that works as a
single enumerator.

e = (1..3).chain([4, 5])
e.to_a #=> [1, 2, 3, 4, 5]

e = (1..3).each + [4, 5]
e.to_a #=> [1, 2, 3, 4, 5]
Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0Like0