https://redmine.ruby-lang.org/https://redmine.ruby-lang.org/favicon.ico?17113305112020-07-08T04:00:04ZRuby Issue Tracking SystemRuby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=864552020-07-08T04:00:04Zsawa (Tsuyoshi Sawada)
<ul></ul><p>What is wrong with the following?</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">].</span><span class="nf">inject</span><span class="p">([</span><span class="mi">0</span><span class="p">]){</span><span class="o">|</span><span class="n">a</span><span class="p">,</span> <span class="n">e</span><span class="o">|</span> <span class="n">a</span> <span class="o"><<</span> <span class="n">a</span><span class="p">.</span><span class="nf">last</span> <span class="o">+</span> <span class="n">e</span><span class="p">}</span> <span class="c1"># => [0, 1, 3, 6]</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">].</span><span class="nf">each_with_object</span><span class="p">([</span><span class="mi">0</span><span class="p">]){</span><span class="o">|</span><span class="n">e</span><span class="p">,</span> <span class="n">a</span><span class="o">|</span> <span class="n">a</span> <span class="o"><<</span> <span class="n">a</span><span class="p">.</span><span class="nf">last</span> <span class="o">+</span> <span class="n">e</span><span class="p">}</span> <span class="c1"># => [0, 1, 3, 6]</span>
</code></pre> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=864602020-07-08T14:51:14Zparker (Parker Finch)
<ul></ul><p>sawa (Tsuyoshi Sawada) wrote in <a href="#note-1">#note-1</a>:</p>
<blockquote>
<p>What is wrong with the following?</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">].</span><span class="nf">inject</span><span class="p">([</span><span class="mi">0</span><span class="p">]){</span><span class="o">|</span><span class="n">a</span><span class="p">,</span> <span class="n">e</span><span class="o">|</span> <span class="n">a</span> <span class="o"><<</span> <span class="n">a</span><span class="p">.</span><span class="nf">last</span> <span class="o">+</span> <span class="n">e</span><span class="p">}</span> <span class="c1"># => [0, 1, 3, 6]</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">].</span><span class="nf">each_with_object</span><span class="p">([</span><span class="mi">0</span><span class="p">]){</span><span class="o">|</span><span class="n">e</span><span class="p">,</span> <span class="n">a</span><span class="o">|</span> <span class="n">a</span> <span class="o"><<</span> <span class="n">a</span><span class="p">.</span><span class="nf">last</span> <span class="o">+</span> <span class="n">e</span><span class="p">}</span> <span class="c1"># => [0, 1, 3, 6]</span>
</code></pre>
</blockquote>
<p>Good question! Using <code>#inject</code> or <code>#each_with_object</code> cannot be done lazily. We need to pass a lazy enumerator <em>after</em> applying the fold-like operation. As a toy example, instead of <code>[1, 2, 3]</code> we can use <code>(1..).lazy</code>:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">inject</span><span class="p">([</span><span class="mi">0</span><span class="p">]){</span><span class="o">|</span><span class="n">a</span><span class="p">,</span> <span class="n">e</span><span class="o">|</span> <span class="n">a</span> <span class="o"><<</span> <span class="n">a</span><span class="p">.</span><span class="nf">last</span> <span class="o">+</span> <span class="n">e</span><span class="p">}</span> <span class="c1"># => infinite loop</span>
<span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">each_with_object</span><span class="p">([</span><span class="mi">0</span><span class="p">]){</span><span class="o">|</span><span class="n">e</span><span class="p">,</span> <span class="n">a</span><span class="o">|</span> <span class="n">a</span> <span class="o"><<</span> <span class="n">a</span><span class="p">.</span><span class="nf">last</span> <span class="o">+</span> <span class="n">e</span><span class="p">}</span> <span class="c1"># => infinite loop</span>
<span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">scan_left</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="o">&</span><span class="p">:</span><span class="o">+</span><span class="p">)</span> <span class="c1"># => Lazy enumerator</span>
</code></pre> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=864762020-07-09T21:33:17ZEregon (Benoit Daloze)
<ul></ul><p>The name <code>scan</code> seems confusing at least, since it has nothing to do with <code>String#scan</code>.<br>
And <code>*_left</code> has no precedent in Ruby.</p>
<p>It seems it's basically recording the history of each block call (+ the initial value), maybe a name expressing that would be good.</p>
<p>Do you have real-world usages you can show?<br>
It seems kind of niche (seems expensive if the collection is large) and can easily be replaced by <code>each</code> + 2 variables (if not lazy at least).</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=864922020-07-10T20:57:27Zparker (Parker Finch)
<ul><li><strong>File</strong> <a href="/attachments/8494">scan_left_example.rb</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/8494/scan_left_example.rb">scan_left_example.rb</a> added</li></ul><blockquote>
<p>The name <code>scan</code> seems confusing at least, since it has nothing to do with <code>String#scan</code>.<br>
And <code>*_left</code> has no precedent in Ruby.</p>
<p>It seems it's basically recording the history of each block call (+ the initial value), maybe a name expressing that would be good.</p>
</blockquote>
<p>Ah sorry, I forgot to mention that the term "scan" is what this operation is typically called in functional programming (see <a href="https://en.wikipedia.org/wiki/Prefix_sum#Scan_higher_order_function" class="external">here</a> for a quick overview). So that's how the name is derived. I agree that the <code>*_left</code> suffix doesn't sound very Ruby-ish, maybe just <code>#scan</code> would be better. But then we <em>really</em> have a conflict with the name of <code>String#scan</code>, thanks for pointing that out! Any thoughts on what this could be called to disambiguate that? Perhaps <code>#accumulate</code>?</p>
<blockquote>
<p>Do you have real-world usages you can show?</p>
</blockquote>
<p>The most significant real-world usage that I've had it for is the one I briefly described above. The "scan" operation is a very good fit when processing streams of data changes. To really concretize it, our use case is in education data. One example is that each day we see if a student is present or absent. We use a scan over that data to count the number of absences that a student has -- a fold (i.e. <code>#inject</code> or <code>#reduce</code>) would be insufficient because the stream is lazy.</p>
<p>To get a small, self-contained example, I wrote up a script that uses it (and shows how other implementations of the behavior could work). That is attached, let me know if there's anything else I can do to show the usefulness here!</p>
<blockquote>
<p>It seems kind of niche (seems expensive if the collection is large) and can easily be replaced by <code>each</code> + 2 variables (if not lazy at least).</p>
</blockquote>
<p>I don't think it's actually that niche! A lot of array algorithms become easier with a <code>scan</code> operation, see <a href="https://medium.com/beauty-date-stories/algorithms-how-prefix-sums-can-help-improving-operations-over-arrays-b1f8e8141668" class="external">this post</a> for an example. And I don't think that it's particularly expensive -- it should be similar in cost to a <code>map</code>.</p>
<p>In fact, scan-like behavior (with laziness) can be implemented with <code>map</code> without <em>too</em> much trouble. (<a href="https://github.com/panorama-ed/scan_left/blob/029d3ac4fa2073afa9d7da5d7932992849c235f2/lib/scan_left.rb#L49" class="external">This</a> is the heart of the ruby implementation.) But assigning to a variable inside of the block passed to <code>map</code> doesn't feel very Ruby-ish to me:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">val</span> <span class="o">=</span> <span class="mi">0</span>
<span class="n">collection</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">x</span><span class="o">|</span> <span class="n">val</span> <span class="o">=</span> <span class="n">val</span> <span class="o">+</span> <span class="n">x</span> <span class="p">}</span>
</code></pre>
<p>feels less natural than something like</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">collection</span><span class="p">.</span><span class="nf">scan</span><span class="p">(</span><span class="o">&</span><span class="p">:</span><span class="o">+</span><span class="p">)</span>
</code></pre>
<p>One of my favorite aspects of Ruby is how easy it is to write in a functional programming style, and including the <code>scan</code> operation would expand the number of use cases covered by functional methods.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=864972020-07-11T05:46:51Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul></ul><p>parker (Parker Finch) wrote in <a href="#note-2">#note-2</a>:</p>
<blockquote>
<p>Good question! Using <code>#inject</code> or <code>#each_with_object</code> cannot be done lazily. We need to pass a lazy enumerator <em>after</em> applying the fold-like operation. As a toy example, instead of <code>[1, 2, 3]</code> we can use <code>(1..).lazy</code>:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">inject</span><span class="p">([</span><span class="mi">0</span><span class="p">]){</span><span class="o">|</span><span class="n">a</span><span class="p">,</span> <span class="n">e</span><span class="o">|</span> <span class="n">a</span> <span class="o"><<</span> <span class="n">a</span><span class="p">.</span><span class="nf">last</span> <span class="o">+</span> <span class="n">e</span><span class="p">}</span> <span class="c1"># => infinite loop</span>
<span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">each_with_object</span><span class="p">([</span><span class="mi">0</span><span class="p">]){</span><span class="o">|</span><span class="n">e</span><span class="p">,</span> <span class="n">a</span><span class="o">|</span> <span class="n">a</span> <span class="o"><<</span> <span class="n">a</span><span class="p">.</span><span class="nf">last</span> <span class="o">+</span> <span class="n">e</span><span class="p">}</span> <span class="c1"># => infinite loop</span>
<span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">scan_left</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="o">&</span><span class="p">:</span><span class="o">+</span><span class="p">)</span> <span class="c1"># => Lazy enumerator</span>
</code></pre>
</blockquote>
<p>That sounds like <code>Enumerator::Lazy</code> needs <code>#inject</code> and <code>#each_with_object</code> methods to me.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=865052020-07-11T17:47:54ZEregon (Benoit Daloze)
<ul></ul><p>parker (Parker Finch) wrote in <a href="#note-4">#note-4</a>:</p>
<p>Thanks for your reply, I think it will help to discuss this issue at the dev meeting.</p>
<p>Maybe <code>prefix_sum</code> or just <code>prefix</code> or something like that would work?<br>
Having <code>sum</code> in it is kind of confusing though as it can be any "operation" not just <code>+</code>-ing numbers, but it seems an official "term" for it (<a href="https://en.wikipedia.org/wiki/Prefix_sum#Scan_higher_order_function" class="external">https://en.wikipedia.org/wiki/Prefix_sum#Scan_higher_order_function</a>).</p>
<blockquote>
<p>But assigning to a variable inside of the block passed to <code>map</code> doesn't feel very Ruby-ish to me:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">val</span> <span class="o">=</span> <span class="mi">0</span>
<span class="n">collection</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">x</span><span class="o">|</span> <span class="n">val</span> <span class="o">=</span> <span class="n">val</span> <span class="o">+</span> <span class="n">x</span> <span class="p">}</span>
</code></pre>
</blockquote>
<p>It's just my opinion, but I see nothing wrong with that (details: it could be <code>val += x</code>).<br>
I'd even go as far as saying <code>each_with_object</code> is often less readable than using a captured variable.<br>
I think "purely functional, not a single re-assigned variable" often introduces significant extra complexity, when Ruby is a language that embraces both functional and imperative programming.<br>
And anyway <code>each_with_object</code> is only useful if mutating some object (typically an Array or Hash).<br>
Again, it's just my opinion :)</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=865062020-07-11T17:49:19ZEregon (Benoit Daloze)
<ul></ul><p>nobu (Nobuyoshi Nakada) wrote in <a href="#note-5">#note-5</a>:</p>
<blockquote>
<p>That sounds like <code>Enumerator::Lazy</code> needs <code>#inject</code> and <code>#each_with_object</code> methods to me.</p>
</blockquote>
<p>A lazy <code>#inject</code> sounds useful.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=865092020-07-12T08:52:27Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul></ul><p>Eregon (Benoit Daloze) wrote in <a href="#note-7">#note-7</a>:</p>
<blockquote>
<p>nobu (Nobuyoshi Nakada) wrote in <a href="#note-5">#note-5</a>:</p>
<blockquote>
<p>That sounds like <code>Enumerator::Lazy</code> needs <code>#inject</code> and <code>#each_with_object</code> methods to me.</p>
</blockquote>
<p>A lazy <code>#inject</code> sounds useful.</p>
</blockquote>
<p>It has a backward incompatibility on the return value.<br>
I’m afraid if it’s acceptable.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=865102020-07-12T11:32:14ZEregon (Benoit Daloze)
<ul></ul><p>nobu (Nobuyoshi Nakada) wrote in <a href="#note-8">#note-8</a>:</p>
<blockquote>
<p>It has a backward incompatibility on the return value.<br>
I’m afraid if it’s acceptable.</p>
</blockquote>
<p>Right, <code>inject</code> might return anything, not necessarily an Array and so returning a "lazy Enumerator" instead will be unwanted in at least some cases.</p>
<p>Using the existing lazy <code>map</code> seems most natural to me:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">val</span> <span class="o">=</span> <span class="mi">0</span>
<span class="nb">p</span> <span class="p">(</span><span class="mi">0</span><span class="o">..</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">x</span><span class="o">|</span> <span class="n">val</span> <span class="o">+=</span> <span class="n">x</span> <span class="p">}.</span><span class="nf">first</span><span class="p">(</span><span class="mi">4</span><span class="p">)</span> <span class="c1"># => [0, 1, 3, 6]</span>
<span class="n">val</span> <span class="o">=</span> <span class="mi">0</span>
<span class="nb">p</span> <span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">+</span> <span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">x</span><span class="o">|</span> <span class="n">val</span> <span class="o">+=</span> <span class="n">x</span> <span class="p">}.</span><span class="nf">first</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span> <span class="c1"># => [0, 1, 3, 6]</span>
</code></pre> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=865192020-07-12T23:51:46Zshyouhei (Shyouhei Urabe)shyouhei@ruby-lang.org
<ul></ul><p>It might be possible to let inject be lazy if we ignore backwards compatibility. But how do we partially evaluate that lazy enumerator then?</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=865452020-07-14T17:50:50Zparker (Parker Finch)
<ul></ul><p>shyouhei (Shyouhei Urabe) wrote in <a href="#note-10">#note-10</a>:</p>
<blockquote>
<p>It might be possible to let inject be lazy if we ignore backwards compatibility. But how do we partially evaluate that lazy enumerator then?</p>
</blockquote>
<p>I think this is the crux of the issue. Because <code>#inject</code> <em>needs</em> to evaluate every element in order to return a meaningful value, it can't be partially evaluated. The "scan" operation allows for partial evaluation.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=865462020-07-14T18:20:54Zparker (Parker Finch)
<ul></ul><p>Eregon (Benoit Daloze) wrote in <a href="#note-6">#note-6</a>:</p>
<blockquote>
<p>Maybe <code>prefix_sum</code> or just <code>prefix</code> or something like that would work?<br>
Having <code>sum</code> in it is kind of confusing though as it can be any "operation" not just <code>+</code>-ing numbers, but it seems an official "term" for it (<a href="https://en.wikipedia.org/wiki/Prefix_sum#Scan_higher_order_function" class="external">https://en.wikipedia.org/wiki/Prefix_sum#Scan_higher_order_function</a>).</p>
</blockquote>
<p>I agree that including "sum" in the name is confusing. I think that "prefix_sum" is just used to describe the sum operation, and if it is generalized to other operations then it is typically called "scan".</p>
<hr>
<p>Eregon (Benoit Daloze) wrote in <a href="#note-9">#note-9</a>:</p>
<blockquote>
<p>Using the existing lazy <code>map</code> seems most natural to me:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">val</span> <span class="o">=</span> <span class="mi">0</span>
<span class="nb">p</span> <span class="p">(</span><span class="mi">0</span><span class="o">..</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">x</span><span class="o">|</span> <span class="n">val</span> <span class="o">+=</span> <span class="n">x</span> <span class="p">}.</span><span class="nf">first</span><span class="p">(</span><span class="mi">4</span><span class="p">)</span> <span class="c1"># => [0, 1, 3, 6]</span>
<span class="n">val</span> <span class="o">=</span> <span class="mi">0</span>
<span class="nb">p</span> <span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">+</span> <span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">x</span><span class="o">|</span> <span class="n">val</span> <span class="o">+=</span> <span class="n">x</span> <span class="p">}.</span><span class="nf">first</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span> <span class="c1"># => [0, 1, 3, 6]</span>
</code></pre>
</blockquote>
<p>It's definitely possible to use <code>map</code>! I think it is simpler to use a scan, especially if the first element needs to be included in the lazy enumerator. For example:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">scan</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="o">&</span><span class="p">:</span><span class="o">+</span><span class="p">)</span>
</code></pre>
<p>would need to be:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">val</span> <span class="o">=</span> <span class="mi">0</span>
<span class="p">[</span><span class="n">val</span><span class="p">].</span><span class="nf">chain</span><span class="p">((</span><span class="mi">1</span><span class="o">..</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">x</span><span class="o">|</span> <span class="n">val</span> <span class="o">+=</span> <span class="n">x</span> <span class="p">})</span>
</code></pre>
<p>because the <code>+</code> method doesn't work with lazy enumerators.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=865562020-07-15T14:32:59ZEregon (Benoit Daloze)
<ul></ul><p>For the name I think just <code>scan</code> would be best then.<br>
And accept the fact it's completely unrelated to <code>String#scan</code>.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=865692020-07-16T05:17:32Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul></ul><p>Is this what you want?</p>
<pre><code>irb(main):001:0> (1..).lazy.enum_for(:inject, 0).map {|a, b| a + b }.take(10).force
=> [1, 3, 6, 10, 15, 21, 28, 36, 45, 55]
</code></pre> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=865822020-07-17T14:29:18Zparker (Parker Finch)
<ul></ul><p>mame (Yusuke Endoh) wrote in <a href="#note-14">#note-14</a>:</p>
<blockquote>
<p>Is this what you want?</p>
<pre><code>irb(main):001:0> (1..).lazy.enum_for(:inject, 0).map {|a, b| a + b }.take(10).force
=> [1, 3, 6, 10, 15, 21, 28, 36, 45, 55]
</code></pre>
</blockquote>
<p>Oh interesting, I hadn't considered that approach! That is very close to the behavior of the scan operation, and might be a good way to implement it. (The only difference in behavior I see is that the first element (<code>0</code>) is not included in the resulting enumerator with this <code>enum_for</code> approach.)</p>
<p>Are you suggesting that we should use that approach instead of implementing a built-in <code>scan</code> method? Or is the example to clarify what the behavior of the <code>scan</code> method would be?</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=865922020-07-18T03:23:02Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul></ul><p>parker (Parker Finch) wrote in <a href="#note-15">#note-15</a>:</p>
<blockquote>
<p>Are you suggesting that we should use that approach instead of implementing a built-in <code>scan</code> method? Or is the example to clarify what the behavior of the <code>scan</code> method would be?</p>
</blockquote>
<p>As it came from the lazy <code>inject</code> approach, the former.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=866192020-07-20T06:02:45Zmatz (Yukihiro Matsumoto)matz@ruby.or.jp
<ul></ul><p>I don't see any real-world use-case for <code>scan_left</code>. Is there any?<br>
In addition, the term <code>scan</code> does not seem to describe the behavior of keeping the past sequence of accumulation.<br>
Although I know the name from the Haskell community, I don't agree with the name.</p>
<p>Matz.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=866572020-07-22T17:23:46Zparker (Parker Finch)
<ul></ul><p>matz (Yukihiro Matsumoto) wrote in <a href="#note-17">#note-17</a>:</p>
<blockquote>
<p>I don't see any real-world use-case for <code>scan_left</code>. Is there any?</p>
</blockquote>
<p>I think there are real-world use cases!</p>
<p>Would you consider converting a history of transactions into a history of account balances a valid use-case? That can be done easily with a scan. For example, if you have <code>transactions = [100, -200, 200]</code> then you can find the history of account balances with <code>transactions.scan_left(0, &:+) # => [0, 100, -100, 100]</code>.</p>
<p>I have described our current use case of <code>scan_left</code> <a href="https://medium.com/building-panorama-education/scan-left-a-lazy-incremental-alternative-to-inject-f6e946f74c00" class="external">here</a>. We use an event-sourced architecture where we scan over a lazy stream of events, building up the state as we go. It is important for our use case that we are able to access past states in addition to the final state.</p>
<p><a href="https://medium.com/beauty-date-stories/algorithms-how-prefix-sums-can-help-improving-operations-over-arrays-b1f8e8141668" class="external">This post</a> shows how it can make repeated subarray operations more efficient -- the example there is that if you have an array of values representing changes over time periods, then you can easily aggregate those into changes over different time periods using a <code>scan</code>. This post does not use <code>scan</code>, but instead has a workaround because <code>scan</code> doesn't exist:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">sums</span> <span class="o">=</span> <span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="n">gains</span><span class="p">.</span><span class="nf">length</span><span class="p">).</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">i</span><span class="o">|</span>
<span class="n">sums</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">sums</span><span class="p">[</span><span class="n">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span> <span class="o">+</span> <span class="n">gains</span><span class="p">[</span><span class="n">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span>
<span class="k">end</span>
</code></pre>
<p>could, if <code>scan</code> was introduced, be replaced with:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">sums</span> <span class="o">=</span> <span class="n">gains</span><span class="p">.</span><span class="nf">scan_left</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="o">&</span><span class="p">:</span><span class="o">+</span><span class="p">)</span>
</code></pre>
<p>Do those use cases seem sufficient?</p>
<hr>
<blockquote>
<p>In addition, the term <code>scan</code> does not seem to describe the behavior of keeping the past sequence of accumulation.<br>
Although I know the name from the Haskell community, I don't agree with the name.</p>
</blockquote>
<p>I agree the name is not ideal. It is easy to get it confused with the idea of a string scanner. As you mentioned, <code>scan</code> is the name used by Haskell, but it is also used by <a href="https://www.scala-lang.org/api/current/scala/Array.html" class="external">Scala</a>, <a href="https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.scan" class="external">Rust</a>, and <a href="https://en.cppreference.com/w/cpp/algorithm/inclusive_scan" class="external">C++</a>. So it is a widely-used term, even if it's not the best one, which makes me think that it might be good to use it.</p>
<p>Alternatively, what do you think about the name <code>accumulate</code>, which <a href="https://reference.wolfram.com/language/ref/Accumulate.html" class="external">Wolfram uses</a>? I think it gets the idea across better than <code>scan</code>.</p>
<p>Are there other names you think should be considered?</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=866622020-07-22T18:12:49ZRubyBugs (A Nonymous)msiegel@riverdaletechinc.com
<ul></ul><p>parker (Parker Finch) wrote in <a href="#note-18">#note-18</a>:</p>
<blockquote>
<p>Are there other names you think should be considered?</p>
</blockquote>
<p>In keeping with the Ruby-ish collection methods that end with "-ect", how about</p>
<ul>
<li>
<strong><code>reflect</code></strong> -- the idea is to contrast with <code>inject</code>, this "reflects" all intermediate states</li>
<li>
<strong><code>project</code></strong> -- the idea is that the original <code>Enumerable</code> is "projected" in a mathematical sense into the plane defined by the stateful function that is passed in</li>
</ul>
<p>(just my 2 cents...)</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=866752020-07-23T03:12:28Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul></ul><p>With <a href="https://github.com/ruby/ruby/pull/3337" class="external">https://github.com/ruby/ruby/pull/3337</a>, you can</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">inject</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="p">:</span><span class="o">+</span><span class="p">).</span><span class="nf">first</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="c1">#=> [0, 1, 3, 6, 10, 15, 21, 28, 36, 45]</span>
</code></pre> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=866892020-07-23T15:39:16Zparker (Parker Finch)
<ul></ul><p>nobu (Nobuyoshi Nakada) wrote in <a href="#note-20">#note-20</a>:</p>
<blockquote>
<p>With <a href="https://github.com/ruby/ruby/pull/3337" class="external">https://github.com/ruby/ruby/pull/3337</a>, you can</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">inject</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="p">:</span><span class="o">+</span><span class="p">).</span><span class="nf">first</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="c1">#=> [0, 1, 3, 6, 10, 15, 21, 28, 36, 45]</span>
</code></pre>
</blockquote>
<p>Thank you for writing that nobu! That is exactly the behavior we need.</p>
<p>I'm worried about the backwards compatibility of changing the behavior of <code>#inject</code> on lazy enumerables. Since right now <code>[1,2,3].lazy.inject(:+) # => 6</code> I think it would be breaking to change that behavior to be <code>[1,2,3].lazy.inject(:+) # => #<Enumerator::Lazy: #<Enumerator::Lazy: [1, 2, 3]>:inject></code>.</p>
<p>I'm also worried about <code>#inject</code> having different behavior for strict and lazy enumerables. I don't think it would be good to have <code>[1,2,3].inject(:+)</code> be different than <code>[1,2,3].lazy.inject(:+).force</code>.</p>
<p>Do you think your implementation could be used for a new method, whether we call it <code>scan</code> or <code>accumulate</code> or <code>project</code> or something else? I think it would be good to have consistent behavior between strict and lazy enumerables.</p>
<hr>
<p><strong>NOTE:</strong> For anyone who is newer to this thread and might have missed it -- you can find this behavior, with some examples, implemented in Ruby <a href="https://github.com/panorama-ed/scan_left/" class="external">here</a>.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=866962020-07-23T21:44:12Zy.annikov (Yakov Annikov)
<ul></ul><p>I've created a PR with the implementation of <code>scan</code> and <code>lazy.scan</code> methods <a href="https://github.com/ruby/ruby/pull/3358" class="external">https://github.com/ruby/ruby/pull/3358</a><br>
Feel free to reject it but any comments are appreciated. I got a lot of fun writing this code and looking at Ruby source code.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=866982020-07-24T02:30:43Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul></ul><p>RubyBugs (A Nonymous) wrote in <a href="#note-19">#note-19</a>:</p>
<blockquote>
<p>In keeping with the Ruby-ish collection methods that end with "-ect", how about</p>
<ul>
<li>
<strong><code>reflect</code></strong> -- the idea is to contrast with <code>inject</code>, this "reflects" all intermediate states</li>
<li>
<strong><code>project</code></strong> -- the idea is that the original <code>Enumerable</code> is "projected" in a mathematical sense into the plane defined by the stateful function that is passed in</li>
</ul>
</blockquote>
<p>I'd like that “this "reflects" all intermediate states” part.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=867222020-07-25T06:20:50Zduerst (Martin Dürst)duerst@it.aoyama.ac.jp
<ul></ul><p>parker (Parker Finch) wrote in <a href="#note-21">#note-21</a>:</p>
<blockquote>
<p>nobu (Nobuyoshi Nakada) wrote in <a href="#note-20">#note-20</a>:</p>
<blockquote>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">inject</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="p">:</span><span class="o">+</span><span class="p">).</span><span class="nf">first</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="c1">#=> [0, 1, 3, 6, 10, 15, 21, 28, 36, 45]</span>
</code></pre>
</blockquote>
</blockquote>
<blockquote>
<p>I'm worried about the backwards compatibility of changing the behavior of <code>#inject</code> on lazy enumerables. Since right now <code>[1,2,3].lazy.inject(:+) # => 6</code> I think it would be breaking to change that behavior to be <code>[1,2,3].lazy.inject(:+) # => #<Enumerator::Lazy: #<Enumerator::Lazy: [1, 2, 3]>:inject></code>.</p>
<p>I'm also worried about <code>#inject</code> having different behavior for strict and lazy enumerables. I don't think it would be good to have <code>[1,2,3].inject(:+)</code> be different than <code>[1,2,3].lazy.inject(:+).force</code>.</p>
</blockquote>
<p>I fully agree. Lazy variants of methods should only differ with respect to laziness, not anything else.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=867232020-07-25T07:49:34Zduerst (Martin Dürst)duerst@it.aoyama.ac.jp
<ul></ul><p>nobu (Nobuyoshi Nakada) wrote in <a href="#note-23">#note-23</a>:</p>
<blockquote>
<p>RubyBugs (A Nonymous) wrote in <a href="#note-19">#note-19</a>:</p>
<blockquote>
<p>In keeping with the Ruby-ish collection methods that end with "-ect", how about</p>
<ul>
<li>
<strong><code>reflect</code></strong> -- the idea is to contrast with <code>inject</code>, this "reflects" all intermediate states</li>
<li>
<strong><code>project</code></strong> -- the idea is that the original <code>Enumerable</code> is "projected" in a mathematical sense into the plane defined by the stateful function that is passed in</li>
</ul>
</blockquote>
<p>I'd like that “this "reflects" all intermediate states” part.</p>
</blockquote>
<p>I think this is way too generic. In the same vein, we could call it "return", because it returns all intermediate states.<br>
Also, I think "project" isn't appropriate, because project is usually associated with a dimension reduction.</p>
<p>The most specific word in the above explanation is "intermediate". I suggest we search more along these lines. An example would be something like "inject_with_intermediates".</p>
<p>BTW, I also checked APL, where '' is used for what we are discussing here, and is called scan. The fact that this is included in APL shows that this is in some sense a core operation. It doesn't appear e.g. in a 1972 manual for IBM APL\360 (<a href="http://www.softwarepreservation.org/projects/apl/Manuals/APL360UsersManuals" class="external">http://www.softwarepreservation.org/projects/apl/Manuals/APL360UsersManuals</a>), which means it may not have been there from the start. But I found some hints in a newer document (1982, <a href="http://www.softwarepreservation.org/projects/apl/Manuals/SharpAPLManualCorrections" class="external">http://www.softwarepreservation.org/projects/apl/Manuals/SharpAPLManualCorrections</a>). My guess is that the name did not come from APL (which is one of the oldest functional programming languages).</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=867252020-07-25T10:12:25Zy.annikov (Yakov Annikov)
<ul></ul><p>y.annikov (Yakov Annikov) wrote in <a href="#note-22">#note-22</a>:</p>
<blockquote>
<p>I've created a PR with the implementation of <code>scan</code> and <code>lazy.scan</code> methods <a href="https://github.com/ruby/ruby/pull/3358" class="external">https://github.com/ruby/ruby/pull/3358</a><br>
Feel free to reject it but any comments are appreciated. I got a lot of fun writing this code and looking at Ruby source code.</p>
</blockquote>
<p>I renamed it to <code>reflect</code> for a while though I like <code>scan</code> more.<br>
Here are examples of the behaviour of <code>reflect</code> method that I've implemented:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># reflect</span>
<span class="p">(</span><span class="mi">0</span><span class="o">...</span><span class="mi">0</span><span class="p">).</span><span class="nf">reflect</span><span class="p">(:</span><span class="o">+</span><span class="p">)</span> <span class="o">=></span> <span class="p">[]</span>
<span class="p">(</span><span class="mi">5</span><span class="o">..</span><span class="mi">10</span><span class="p">).</span><span class="nf">reflect</span><span class="p">(:</span><span class="o">+</span><span class="p">)</span> <span class="o">=></span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">18</span><span class="p">,</span> <span class="mi">26</span><span class="p">,</span> <span class="mi">35</span><span class="p">,</span> <span class="mi">45</span><span class="p">]</span>
<span class="p">(</span><span class="mi">5</span><span class="o">..</span><span class="mi">10</span><span class="p">).</span><span class="nf">reflect</span> <span class="p">{</span> <span class="o">|</span><span class="n">sum</span><span class="p">,</span> <span class="n">n</span><span class="o">|</span> <span class="n">sum</span> <span class="o">+</span> <span class="n">n</span> <span class="p">}</span> <span class="o">=></span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">18</span><span class="p">,</span> <span class="mi">26</span><span class="p">,</span> <span class="mi">35</span><span class="p">,</span> <span class="mi">45</span><span class="p">]</span>
<span class="p">(</span><span class="mi">5</span><span class="o">..</span><span class="mi">10</span><span class="p">).</span><span class="nf">reflect</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="p">:</span><span class="o">*</span><span class="p">)</span> <span class="o">=></span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">30</span><span class="p">,</span> <span class="mi">210</span><span class="p">,</span> <span class="mi">1680</span><span class="p">,</span> <span class="mi">15120</span><span class="p">,</span> <span class="mi">151200</span><span class="p">]</span>
<span class="p">(</span><span class="mi">5</span><span class="o">..</span><span class="mi">10</span><span class="p">).</span><span class="nf">reflect</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="p">{</span> <span class="o">|</span><span class="n">product</span><span class="p">,</span> <span class="n">n</span><span class="o">|</span> <span class="n">product</span> <span class="o">*</span> <span class="n">n</span> <span class="p">}</span> <span class="o">=></span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">30</span><span class="p">,</span> <span class="mi">210</span><span class="p">,</span> <span class="mi">1680</span><span class="p">,</span> <span class="mi">15120</span><span class="p">,</span> <span class="mi">151200</span><span class="p">]</span>
</code></pre>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="c1"># lazy.reflect</span>
<span class="p">(</span><span class="mi">0</span><span class="o">..</span><span class="no">Float</span><span class="o">::</span><span class="no">INFINITY</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">reflect</span><span class="p">(:</span><span class="o">+</span><span class="p">).</span><span class="nf">first</span><span class="p">(</span><span class="mi">4</span><span class="p">)</span> <span class="o">=></span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">6</span><span class="p">]</span>
<span class="p">(</span><span class="mi">0</span><span class="o">..</span><span class="no">Float</span><span class="o">::</span><span class="no">INFINITY</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">reflect</span><span class="p">(:</span><span class="o">+</span><span class="p">).</span><span class="nf">first</span><span class="p">(</span><span class="mi">4</span><span class="p">)</span> <span class="o">=></span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">]</span>
<span class="n">enum</span> <span class="o">=</span> <span class="p">(</span><span class="mi">5</span><span class="o">..</span><span class="mi">10</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">reflect</span><span class="p">(:</span><span class="o">+</span><span class="p">)</span> <span class="o">=></span> <span class="c1">#<Enumerator::Lazy: #<Enumerator::Lazy: 5..10>:reflect(:+)></span>
<span class="mi">6</span><span class="p">.</span><span class="nf">times</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="n">enum</span><span class="p">.</span><span class="nf">next</span> <span class="p">}</span> <span class="o">=></span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">18</span><span class="p">,</span> <span class="mi">26</span><span class="p">,</span> <span class="mi">35</span><span class="p">,</span> <span class="mi">45</span><span class="p">]</span>
<span class="n">enum</span> <span class="o">=</span> <span class="p">(</span><span class="mi">5</span><span class="o">..</span><span class="mi">10</span><span class="p">).</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">reflect</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="p">:</span><span class="o">*</span><span class="p">)</span> <span class="o">=></span> <span class="c1">#<Enumerator::Lazy: #<Enumerator::Lazy: 5..10>:reflect(1, :*)></span>
<span class="mi">7</span><span class="p">.</span><span class="nf">times</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="n">enum</span><span class="p">.</span><span class="nf">next</span> <span class="p">}</span> <span class="o">=></span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">30</span><span class="p">,</span> <span class="mi">210</span><span class="p">,</span> <span class="mi">1680</span><span class="p">,</span> <span class="mi">15120</span><span class="p">,</span> <span class="mi">151200</span><span class="p">]</span>
</code></pre>
<p><strong>UPD:</strong> Some fixes according to <a class="user active user-mention" href="https://redmine.ruby-lang.org/users/50">@duerst (Martin Dürst)</a> comments.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=867702020-07-28T17:42:36ZRubyBugs (A Nonymous)msiegel@riverdaletechinc.com
<ul></ul><p>To return to naming for a moment. Did I understand an interest may exist in naming that reflects a connection or duality with <code>inject</code>?</p>
<p>If so, might the <strong>scan</strong> operation find a better Ruby name in <strong>interject</strong>?</p>
<p>The idea would be to say to new learners, with zero prior context: “the <code>interject</code> method has the same form as inject, but returns the stream of <strong>intermediate</strong> values”?</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=867832020-07-28T23:34:27Zduerst (Martin Dürst)duerst@it.aoyama.ac.jp
<ul></ul><p>RubyBugs (A Nonymous) wrote in <a href="#note-27">#note-27</a>:</p>
<blockquote>
<p>To return to naming for a moment. Did I understand an interest may exist in naming that reflects a connection or duality with <code>inject</code>?</p>
</blockquote>
<p>I guess that wouldn't be a bad idea.</p>
<blockquote>
<p>If so, might the <strong>scan</strong> operation find a better Ruby name in <strong>interject</strong>?</p>
<p>The idea would be to say to new learners, with zero prior context: “the <code>interject</code> method has the same form as inject, but returns the stream of <strong>intermediate</strong> values”?</p>
</blockquote>
<p>My most immediate impression of <code>interject</code> is that it would add something between <code>Array</code> elements (e.g. <code>[5..8].interject(0)</code> #→ [5, 0, 6, 0, 7, 0, 8] or so). But the parallel to <code>inject</code> is appealing. And <code>inject</code> itself isn't really that much of a descriptive name, either.</p>
<p>We would have to check whether <code>interject</code> is used in other languages, and for what.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=868522020-07-30T19:16:48Zparker (Parker Finch)
<ul></ul><p>I'd like to sum up where we're at with this discussion. Let me know if you disagree with my interpretation here!</p>
<ol>
<li>There is some support for the idea of adding this method.</li>
<li>We should avoid changing the behavior of <code>#inject</code> on lazy enumerables, since it would be confusing for the lazy version of a method to have different behavior than the strict version.</li>
<li>There is concern around the name of this method. Possibilities that have been discussed are:</li>
</ol>
<ul>
<li>
<code>#scan</code>: This name is not very intuitive for the operation and also has other meanings. However, it is used in many other programming languages.</li>
<li>
<code>#accumulate</code>: This would have my vote at the moment. It suggests that something is accumulated throughout the iteration of the collection. However, it is still a very generic term.</li>
<li>
<code>#reflect</code>, <code>#project</code>, <code>#interject</code>: These end in <code>-ect</code> as many Ruby/Smalltalk methods on collections do. However, they have little meaning related to the operation at hand. I agree that it would be good to show the duality between <code>#inject</code> and this operation, but I'm concerned about using an unrelated word.</li>
</ul>
<p>I think the next steps here are to get confirmation from <a class="user active user-mention" href="https://redmine.ruby-lang.org/users/13">@matz (Yukihiro Matsumoto)</a> as to whether or not he thinks this would be a good method to add to Ruby (I described some use cases in <a href="#note-18">#note-18</a>) and to decide on a name. Does that make sense?</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=868592020-07-31T05:47:56Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul></ul><p>I found a word <a href="https://www.dictionary.com/browse/traject" class="external">traject</a>, means to transport, transmit, or transpose.<br>
It may (or may not) imply <a href="https://www.dictionary.com/browse/trajectory" class="external">trajectory</a>...</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=870042020-08-10T16:47:16Zparker (Parker Finch)
<ul></ul><p>nobu (Nobuyoshi Nakada) wrote in <a href="#note-30">#note-30</a>:</p>
<blockquote>
<p>I found a word <a href="https://www.dictionary.com/browse/traject" class="external">traject</a>, means to transport, transmit, or transpose.<br>
It may (or may not) imply <a href="https://www.dictionary.com/browse/trajectory" class="external">trajectory</a>...</p>
</blockquote>
<p>That's an interesting word that I don't know. It looks like it's archaic, so I don't think that it has much meaning anymore. That's kind of nice, since there's not a conflicting definition that people will have in their heads. However, it also doesn't have a meaning that describes what the method does.</p>
<p>It does have a nice symmetry with <code>inject</code> though! Curious if others have thoughts?</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=871482020-08-21T14:22:36Zparker (Parker Finch)
<ul></ul><p>I'd like to propose that we name this method <code>#accumulate</code>. <a class="user active user-mention" href="https://redmine.ruby-lang.org/users/13">@matz (Yukihiro Matsumoto)</a> do you think that is an acceptable name?</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=872812020-08-29T11:41:46ZDan0042 (Daniel DeLorme)
<ul></ul><p><code>#accumulate</code> is good, but since <a href="https://stackoverflow.com/q/1475808" class="external">this question</a> linked in the OP was asking for the "cumulative sum" ... what about <code>#cumulative</code> ?</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">].</span><span class="nf">cumulative</span><span class="p">.</span><span class="nf">sum</span> <span class="c1">#=> [1,3,6]</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">].</span><span class="nf">cumulative</span><span class="p">.</span><span class="nf">inject</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="p">:</span><span class="o">*</span><span class="p">)</span> <span class="c1">#=> [[5], [5,10], [5,10,30]]</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">].</span><span class="nf">cumulative</span><span class="p">.</span><span class="nf">select</span><span class="p">(</span><span class="o">&</span><span class="ss">:odd?</span><span class="p">)</span> <span class="c1">#=> [[1], [1,3]]</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">].</span><span class="nf">cumulative</span><span class="p">.</span><span class="nf">map</span><span class="p">{</span> <span class="n">_1</span> <span class="o">*</span> <span class="mi">3</span> <span class="p">}</span> <span class="c1">#=> [[3], [3,6], [3,6,9]]</span>
</code></pre> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=874072020-09-03T14:57:17Zparker (Parker Finch)
<ul></ul><p>Dan0042 (Daniel DeLorme) wrote in <a href="#note-33">#note-33</a>:</p>
<blockquote>
<p>... what about <code>#cumulative</code> ?</p>
</blockquote>
<p>Oh that's interesting! I had leapt straight to verbs since that tends to be the pattern for methods that transform enumerables, but the examples of <code>[1,2,3].cumulative.some_other_method</code> are compelling and make sense in a "it returns a cumulative enumerator" way.</p>
<p>I don't think it fits quite as well when a block is passed though; <code>[1,2,3].cumulative(0, &:+)</code> doesn't read as naturally to me as the imperative <code>#accumulate</code>.</p>
<p>I think either option is good -- although since there's a pattern of enumerable methods being named imperatively (e.g. <code>#map</code>, <code>#select</code>, <code>#inject</code>, <code>#drop</code>) I still slightly lean toward the <code>#accumulate</code> option.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=884072020-11-10T14:06:29Zy.annikov (Yakov Annikov)
<ul></ul><p>parker (Parker Finch) wrote in <a href="#note-34">#note-34</a>:</p>
<blockquote>
<p>I still slightly lean toward the <code>#accumulate</code> option.</p>
</blockquote>
<p>What about <code>#cumulate</code>? Does it sound natural?</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=888422020-11-30T14:38:02Zparker (Parker Finch)
<ul></ul><p>y.annikov (Yakov Annikov) wrote in <a href="#note-35">#note-35</a>:</p>
<blockquote>
<p>What about <code>#cumulate</code>? Does it sound natural?</p>
</blockquote>
<p>I'm not as familiar with the verb "cumulate", so it doesn't sound as natural to me as "accumulate". But that being said, I don't think there's a huge difference between the two words, so either one could work!</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=915222021-04-13T14:51:16Zparker (Parker Finch)
<ul></ul><p>I like <code>#accumulate</code>, and thank you <a class="user active user-mention" href="https://redmine.ruby-lang.org/users/4">@nobu (Nobuyoshi Nakada)</a> for an implementation <a href="https://github.com/ruby/ruby/pull/3361" class="external">here</a>!</p>
<p>Is there anything I can do to help us come to a decision on this one?</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=915232021-04-13T14:56:57Zparker (Parker Finch)
<ul><li><strong>Subject</strong> changed from <i>Enumerable#scan_left</i> to <i>Enumerable#accumulate</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/91523/diff?detail_id=59766">diff</a>)</li></ul> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=915932021-04-17T08:15:55Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul></ul><p>This ticket was discussed in the dev meeting, but no conclusion was reached.</p>
<p>BTW, Mathematica's "Accumulate" is different from this proposal. It is not general but specific to Plus.</p>
<p><a href="https://reference.wolfram.com/language/ref/Accumulate.html" class="external">https://reference.wolfram.com/language/ref/Accumulate.html</a></p>
<blockquote>
<p>Accumulate[list] is effectively equivalent to FoldList[Plus,list].</p>
</blockquote>
<p>Thus, FoldList is more suitable name than Accumulate in this case.</p>
<p><a href="https://reference.wolfram.com/language/ref/FoldList.html" class="external">https://reference.wolfram.com/language/ref/FoldList.html</a></p>
<p>The use case explained in #18 is not so convincing that it deserves a built-in feature. To make something built-in, one should show that it is so frequently written, and/or that it is difficult to work around. We are not sure that retaining a historical context is a very frequent code pattern (<a class="user active user-mention" href="https://redmine.ruby-lang.org/users/482">@mrkn (Kenta Murata)</a> said that cumulative sum is actually often used in mathematics, but other use case than cumulative sum was not clear). And Eregon's workaround in #9 looks very simple (maybe even simpler than scan_left).</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=917092021-04-27T09:05:08Zduerst (Martin Dürst)duerst@it.aoyama.ac.jp
<ul></ul><p>mame (Yusuke Endoh) wrote in <a href="#note-39">#note-39</a>:</p>
<blockquote>
<p>This ticket was discussed in the dev meeting, but no conclusion was reached.</p>
<p>BTW, Mathematica's "Accumulate" is different from this proposal. It is not general but specific to Plus.</p>
<p><a href="https://reference.wolfram.com/language/ref/Accumulate.html" class="external">https://reference.wolfram.com/language/ref/Accumulate.html</a></p>
<blockquote>
<p>Accumulate[list] is effectively equivalent to FoldList[Plus,list].</p>
</blockquote>
</blockquote>
<p>That's true.</p>
<blockquote>
<p>Thus, FoldList is more suitable name than Accumulate in this case.</p>
<p><a href="https://reference.wolfram.com/language/ref/FoldList.html" class="external">https://reference.wolfram.com/language/ref/FoldList.html</a></p>
</blockquote>
<p>I don't thinks FoldList would be appropriate in Ruby. I think if we use that name, it would be <code>fold_list</code>. But we are not dealing with lists, we are dealing with Arrays and Enumerables. But we wouldn't use <code>fold_array</code>, and reducing this to <code>fold</code> only wouldn't be appropriate, because fold is a much more general operation (in general, it's closer to Ruby's <code>inject</code>).</p>
<blockquote>
<p>The use case explained in #18 is not so convincing that it deserves a built-in feature. To make something built-in, one should show that it is so frequently written, and/or that it is difficult to work around. We are not sure that retaining a historical context is a very frequent code pattern (<a class="user active user-mention" href="https://redmine.ruby-lang.org/users/482">@mrkn (Kenta Murata)</a> said that cumulative sum is actually often used in mathematics, but other use case than cumulative sum was not clear).</p>
</blockquote>
<p>Even if it's only cumulative sum, that's still quite useful. And if the cumulative sum is the main use case, <code>accumulate</code> may not be such a bad name. Even if this were e.g. for products, <code>accumulate</code> would still work in some sense. Another name might be <code>trace</code>, i.e. in the sense of tracing the evolution of a calculation.</p>
<p>As mentioned above, that's very relevant in a lazy context. A good example might be a bank account with a stream of transfers.</p>
<blockquote>
<p>And Eregon's workaround in #9 looks very simple (maybe even simpler than scan_left).</p>
</blockquote>
<p>I agree that this doesn't feel very Ruby-ish. For me it's not so much the assignment in the block, but the initialization of a variable outside the block.</p> Ruby master - Feature #17016: Enumerable#accumulatehttps://redmine.ruby-lang.org/issues/17016?journal_id=917102021-04-27T15:14:49ZRubyBugs (A Nonymous)msiegel@riverdaletechinc.com
<ul></ul><p>Thanks everyone continuing to discuss whether to add this method to the Ruby lazy Enumerable!</p>
<p>In case it is helpful, please permit me to clarify that this method (and the functional programming pattern it represents) is of <strong>practical</strong>, rather than theoretical benefit.</p>
<p>This method underlies the system presented in the following conference talk <em>ETL and Event Sourcing</em>, which daily rebuilds all system state by re-processing enumerations of the history of data extracted from external systems:</p>
<ul>
<li>Slides: <a href="https://www.slideshare.net/ms-tg/etl-and-event-sourcing" class="external">https://www.slideshare.net/ms-tg/etl-and-event-sourcing</a>
</li>
<li>Video (part 1): <a href="https://www.dropbox.com/s/vibkr2edqmtid9n/047_etl_and_event_sourcing_marc_siegel_panorama_education_part_1.mp4?dl=0" class="external">https://www.dropbox.com/s/vibkr2edqmtid9n/047_etl_and_event_sourcing_marc_siegel_panorama_education_part_1.mp4?dl=0</a>
</li>
<li>Video (part 2): <a href="https://www.dropbox.com/s/o6bwxymrkmbepgr/048_etl_and_event_sourcing_marc_siegel_panorama_education_part_2.mp4?dl=0" class="external">https://www.dropbox.com/s/o6bwxymrkmbepgr/048_etl_and_event_sourcing_marc_siegel_panorama_education_part_2.mp4?dl=0</a>
</li>
</ul>
<p>The implementation of this method we use is published here:</p>
<ul>
<li>Rubygems: <a href="https://rubygems.org/gems/scan_left" class="external">https://rubygems.org/gems/scan_left</a>
</li>
<li>Github: <a href="https://github.com/panorama-ed/scan_left/" class="external">https://github.com/panorama-ed/scan_left/</a>
</li>
</ul>
<p>A blog post introducing the gem and discussing its usage: <a href="https://medium.com/building-panorama-education/scan-left-a-lazy-incremental-alternative-to-inject-f6e946f74c00" class="external">https://medium.com/building-panorama-education/scan-left-a-lazy-incremental-alternative-to-inject-f6e946f74c00</a></p>
<p>Sincere apologies if this additional context is redundant or unnecessary. My intent in presenting this context is just to clarify that this is practical Ruby code extracted from a production system, rather than a purely theoretical matter of interest.</p>
<p>Thanks again!</p>