Ruby Issue Tracking System: Issueshttps://redmine.ruby-lang.org/https://redmine.ruby-lang.org/favicon.ico?17113305112021-11-30T22:52:11ZRuby Issue Tracking System
Redmine Ruby master - Misc #18371 (Assigned): Release branches (release information in general)https://redmine.ruby-lang.org/issues/183712021-11-30T22:52:11Ztenderlovemaking (Aaron Patterson)tenderlove@ruby-lang.org
<p>Hi,</p>
<p>I was trying to learn about Ruby's release process. I noticed that we don't create a release branch until the final version is shipped. Is there a reason we don't create the release branch when the first preview is shipped? The reason I'm asking is because I'm worried about merging things to master after the first preview. Do we have any documentation on the release process? (I was searching and couldn't find much info, but maybe I didn't search correctly)</p>
<p>Thanks!</p> Ruby master - Feature #17638 (Assigned): Support backtracing with the libbacktrace libraryhttps://redmine.ruby-lang.org/issues/176382021-02-17T13:03:12Zxtkoba (Tee KOBAYASHI)
<p>It seems that Ruby's current <code>addr2line.c</code> has trouble with the DWARF 5 debugging format (Bug <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: DWARF5 support? (Closed)" href="https://redmine.ruby-lang.org/issues/17585">#17585</a>).</p>
<p>I propose that there be an option to use the libbacktrace library instead of <code>addr2line.c</code>.</p>
<p>A patch is attached for that. When using libbacktrace, the C level backtrace information looks as follows:</p>
<pre><code>-- C level backtrace information -------------------------------------------
0x7f0cc2b3b372 rb_vm_bugreport
/var/tmp/ruby.build/ruby-devel-x86_64/vm_dump.c:1047
0x7f0cc291e188 rb_bug_for_fatal_signal
/var/tmp/ruby.build/ruby-devel-x86_64/error.c:801
0x7f0cc2a8a137 sigsegv
/var/tmp/ruby.build/ruby-devel-x86_64/signal.c:960
0x7f0cc281a9bf ???
???:0
0x7f0cc247ddf7 ???
???:0
0x7f0cc2a8990d rb_f_kill
/var/tmp/ruby.build/ruby-devel-x86_64/signal.c:481
0x7f0cc2a2e684 proc_rb_f_kill
/var/tmp/ruby.build/ruby-devel-x86_64/process.c:8604
0x7f0cc2b0f2a4 ractor_safe_call_cfunc_m1
/var/tmp/ruby.build/ruby-devel-x86_64/vm_insnhelper.c:2734
0x7f0cc2b0fecb vm_call_cfunc_with_frame
/var/tmp/ruby.build/ruby-devel-x86_64/vm_insnhelper.c:2924
0x7f0cc2b10088 vm_call_cfunc
/var/tmp/ruby.build/ruby-devel-x86_64/vm_insnhelper.c:2945
0x7f0cc2b11b3b vm_call_method_each_type
/var/tmp/ruby.build/ruby-devel-x86_64/vm_insnhelper.c:3414
0x7f0cc2b11fde vm_call_method
/var/tmp/ruby.build/ruby-devel-x86_64/vm_insnhelper.c:3507
0x7f0cc2b121ca vm_call_general
/var/tmp/ruby.build/ruby-devel-x86_64/vm_insnhelper.c:3550
0x7f0cc2b144e7 vm_sendish
/var/tmp/ruby.build/ruby-devel-x86_64/vm_insnhelper.c:4525
0x7f0cc2b1b196 vm_exec_core
/var/tmp/ruby.build/ruby-devel-x86_64/insns.def:789
0x7f0cc2b308f5 rb_vm_exec
/var/tmp/ruby.build/ruby-devel-x86_64/vm.c:2162
0x7f0cc2b316e8 rb_iseq_eval_main
/var/tmp/ruby.build/ruby-devel-x86_64/vm.c:2419
0x7f0cc292778d rb_ec_exec_node
/var/tmp/ruby.build/ruby-devel-x86_64/eval.c:317
0x7f0cc29278d3 ruby_run_node
/var/tmp/ruby.build/ruby-devel-x86_64/eval.c:375
0x55ad53234234 main
./main.c:47
0x7f0cc2468e59 ???
???:0
0x55ad532340f9 ???
???:0
0xffffffffffffffff ???
???:0
</code></pre>
<p>The source code of libbacktrace is available from: <a href="https://github.com/ianlancetaylor/libbacktrace" class="external">https://github.com/ianlancetaylor/libbacktrace</a></p> Ruby master - Feature #17111 (Assigned): Improve performance of Net::HTTPHeader#set_form by 40%https://redmine.ruby-lang.org/issues/171112020-08-10T04:09:30Ztonytonyjan (Weihang Jian)tonytonyjan@gmail.com
<a name="diff"></a>
<h2 >diff<a href="#diff" class="wiki-anchor">¶</a></h2>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/lib/net/http/header.rb b/lib/net/http/header.rb
index a8901e7..3f1a008 100644
</span><span class="gd">--- a/lib/net/http/header.rb
</span><span class="gi">+++ b/lib/net/http/header.rb
</span><span class="p">@@ -475,9 +475,8 @@</span> def set_form(params, enctype='application/x-www-form-urlencoded', formopt={})
@body = nil
@body_stream = nil
@form_option = formopt
<span class="gd">- case enctype
- when /\Aapplication\/x-www-form-urlencoded\z/i,
- /\Amultipart\/form-data\z/i
</span><span class="gi">+ case enctype.downcase
+ when 'application/x-www-form-urlencoded', 'multipart/form-data'
</span> self.content_type = enctype
else
raise ArgumentError, "invalid enctype: #{enctype}"
</code></pre>
<a name="benchmark"></a>
<h2 >benchmark<a href="#benchmark" class="wiki-anchor">¶</a></h2>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">require</span> <span class="s1">'benchmark'</span>
<span class="nb">require</span> <span class="s1">'net/http'</span>
<span class="k">module</span> <span class="nn">Net::HTTPHeader</span>
<span class="k">def</span> <span class="nf">set_form2</span><span class="p">(</span><span class="n">params</span><span class="p">,</span> <span class="n">enctype</span> <span class="o">=</span> <span class="s1">'application/x-www-form-urlencoded'</span><span class="p">,</span> <span class="n">formopt</span> <span class="o">=</span> <span class="p">{})</span>
<span class="vi">@body_data</span> <span class="o">=</span> <span class="n">params</span>
<span class="vi">@body</span> <span class="o">=</span> <span class="kp">nil</span>
<span class="vi">@body_stream</span> <span class="o">=</span> <span class="kp">nil</span>
<span class="vi">@form_option</span> <span class="o">=</span> <span class="n">formopt</span>
<span class="k">case</span> <span class="n">enctype</span><span class="p">.</span><span class="nf">downcase</span>
<span class="k">when</span> <span class="s1">'application/x-www-form-urlencoded'</span><span class="p">,</span> <span class="s1">'multipart/form-data'</span>
<span class="nb">self</span><span class="p">.</span><span class="nf">content_type</span> <span class="o">=</span> <span class="n">enctype</span>
<span class="k">else</span>
<span class="k">raise</span> <span class="no">ArgumentError</span><span class="p">,</span> <span class="s2">"invalid enctype: </span><span class="si">#{</span><span class="n">enctype</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">n</span> <span class="o">=</span> <span class="mi">500_000</span>
<span class="n">request</span> <span class="o">=</span> <span class="no">Net</span><span class="o">::</span><span class="no">HTTP</span><span class="o">::</span><span class="no">Post</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'/'</span><span class="p">)</span>
<span class="no">Benchmark</span><span class="p">.</span><span class="nf">bm</span> <span class="k">do</span> <span class="o">|</span><span class="n">x</span><span class="o">|</span>
<span class="no">GC</span><span class="p">.</span><span class="nf">disable</span>
<span class="n">x</span><span class="p">.</span><span class="nf">report</span> <span class="p">{</span> <span class="n">n</span><span class="p">.</span><span class="nf">times</span> <span class="p">{</span> <span class="n">request</span><span class="p">.</span><span class="nf">set_form</span> <span class="p">[]</span> <span class="p">}</span> <span class="p">}</span>
<span class="n">x</span><span class="p">.</span><span class="nf">report</span> <span class="p">{</span> <span class="n">n</span><span class="p">.</span><span class="nf">times</span> <span class="p">{</span> <span class="n">request</span><span class="p">.</span><span class="nf">set_form2</span> <span class="p">[]</span> <span class="p">}</span> <span class="p">}</span>
<span class="k">end</span>
</code></pre>
<pre><code> user system total real
0.777054 0.101768 0.878822 ( 0.880472)
0.539860 0.088957 0.628817 ( 0.630178)
</code></pre>
<p>I don't see any test for <code>#set_form</code> in <code>test/net/http/test_httpheader.rb</code>, let me know if I need to add more tests, thanks!</p> Ruby master - Feature #16559 (Open): Net::HTTP#request injects "Connection: close" header if #sta...https://redmine.ruby-lang.org/issues/165592020-01-24T03:50:42Zf3ndot (Justin Bull)
<p>Hello,</p>
<p>There appears to be a bug in Net::HTTP#request (and thus #get, #post, etc.) on an instance that isn't explicitly started by the programmer (by invoking #start first, or by executing #request inside a block passed to #start).</p>
<p>Inspecting the source code, it reveals #request will recursively call itself inside a #start block if #started? is false. This is great and as I'd expect.</p>
<p>However in production and in a test setup I'm observing TCP socket connections on the server-side in the "TIME_WAIT" state, indicating the socket was never properly closed. Conversely, explicitly running #request inside a #start block yields no such behaviour.</p>
<p>Consider the following setup, assuming you have docker:</p>
<pre><code>docker run --rm -it -p 8080:80/tcp --user root ubuntu
apt-get update && apt-get install net-tools watch nginx
service nginx start
watch 'netstat -tunapl'
</code></pre>
<p>Running this on your host machine:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">net</span> <span class="o">=</span> <span class="no">Net</span><span class="o">::</span><span class="no">HTTP</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'localhost'</span><span class="p">,</span> <span class="mi">8080</span><span class="p">)</span>
<span class="mi">50</span><span class="p">.</span><span class="nf">times</span> <span class="p">{</span> <span class="n">net</span><span class="p">.</span><span class="nf">get</span><span class="p">(</span><span class="s1">'/'</span><span class="p">)</span> <span class="p">}</span> <span class="c1"># is bad</span>
</code></pre>
<p>Will spawn 50 TCP connections on the server, and will all have on TIME_WAIT for 60 seconds (different *nix OSes have different times):</p>
<pre><code>Every 2.0s: netstat -tunapl
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 791/nginx: master p
tcp 0 0 172.17.0.2:80 172.17.0.1:60772 TIME_WAIT -
tcp 0 0 172.17.0.2:80 172.17.0.1:60732 TIME_WAIT -
tcp 0 0 172.17.0.2:80 172.17.0.1:60812 TIME_WAIT -
tcp 0 0 172.17.0.2:80 172.17.0.1:60778 TIME_WAIT -
...
</code></pre>
<p>However running any of these incantations have no such result:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="mi">50</span><span class="p">.</span><span class="nf">times</span> <span class="p">{</span> <span class="no">Net</span><span class="o">::</span><span class="no">HTTP</span><span class="p">.</span><span class="nf">get</span><span class="p">(</span><span class="no">URI</span><span class="p">(</span><span class="s1">'http://localhost:8080/'</span><span class="p">))</span> <span class="p">}</span> <span class="c1"># is OK</span>
</code></pre>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">net</span> <span class="o">=</span> <span class="no">Net</span><span class="o">::</span><span class="no">HTTP</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'localhost'</span><span class="p">,</span> <span class="mi">8080</span><span class="p">)</span>
<span class="n">net</span><span class="p">.</span><span class="nf">start</span>
<span class="mi">50</span><span class="p">.</span><span class="nf">times</span> <span class="p">{</span> <span class="n">net</span><span class="p">.</span><span class="nf">get</span><span class="p">(</span><span class="s1">'/'</span><span class="p">)</span> <span class="p">}</span> <span class="c1"># is OK</span>
<span class="n">net</span><span class="p">.</span><span class="nf">finish</span>
</code></pre>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">net</span> <span class="o">=</span> <span class="no">Net</span><span class="o">::</span><span class="no">HTTP</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'localhost'</span><span class="p">,</span> <span class="mi">8080</span><span class="p">)</span>
<span class="mi">50</span><span class="p">.</span><span class="nf">times</span> <span class="p">{</span> <span class="n">net</span><span class="p">.</span><span class="nf">start</span> <span class="p">{</span> <span class="n">net</span><span class="p">.</span><span class="nf">get</span><span class="p">(</span><span class="s1">'/'</span><span class="p">)</span> <span class="p">}</span> <span class="p">}</span> <span class="c1"># is OK</span>
</code></pre>
<p>These TIME_WAIT connections matter because a server receiving many HTTP requests from clients using Net::HTTP in this fashion (as Faraday does<a href="https://github.com/lostisland/faraday/pull/1117" class="external">1</a>) the server will begin to oversaturate and timeout past a particular scale.</p>
<p>I've tested and reproduced this in 2.7 and 2.6.</p> Ruby master - Feature #15940 (Open): Coerce symbols internal fstrings in UTF8 rather than ASCII t...https://redmine.ruby-lang.org/issues/159402019-06-19T15:00:12Zbyroot (Jean Boussier)byroot@ruby-lang.org
<p>Patch: <a href="https://github.com/ruby/ruby/pull/2242" class="external">https://github.com/ruby/ruby/pull/2242</a></p>
<p>It's not uncommon for symbols to have literal string counterparts, e.g.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">class</span> <span class="nc">User</span>
<span class="nb">attr_accessor</span> <span class="ss">:name</span>
<span class="k">def</span> <span class="nf">as_json</span>
<span class="p">{</span> <span class="s1">'name'</span> <span class="o">=></span> <span class="nb">name</span> <span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
<p>Since the default source encoding is UTF-8, and that symbols coerce their internal fstring to ASCII when possible, the above snippet will actually keep two instances of <code>"name"</code> in the fstring registry. One in ASCII, the other in UTF-8.</p>
<p>Considering that UTF-8 is a strict superset of ASCII, storing the symbols fstrings as UTF-8 instead makes no significant difference, but allows in most cases to reuse the equivalent string literals.</p>
<p>The only notable behavioral change is <code>Symbol#to_s</code>.</p>
<p>Previously <code>:name.to_s.encoding</code> would be <code>#<Encoding:US-ASCII></code>.<br>
After this patch it's <code>#<Encoding:UTF-8></code>. I can't foresee any significant compatibility impact of this change on existing code.</p>
<p>However, there are several ruby specs asserting this behavior, but I don't know if they can be changed or not: <a href="https://github.com/ruby/spec/commit/a73a1c11f13590dccb975ba4348a04423c009453" class="external">https://github.com/ruby/spec/commit/a73a1c11f13590dccb975ba4348a04423c009453</a></p>
<p>If this specification is impossible to change, then we could consider changing the encoding of the String returned by <code>Symbol#to_s</code>, e.g in ruby pseudo code:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">def</span> <span class="nf">to_s</span>
<span class="n">str</span> <span class="o">=</span> <span class="n">fstr</span><span class="p">.</span><span class="nf">dup</span>
<span class="n">str</span><span class="p">.</span><span class="nf">force_encoding</span><span class="p">(</span><span class="no">Encoding</span><span class="o">::</span><span class="no">ASCII</span><span class="p">)</span> <span class="k">if</span> <span class="n">str</span><span class="p">.</span><span class="nf">ascii_only?</span>
<span class="n">str</span>
<span class="k">end</span>
</code></pre> Ruby master - Feature #15931 (Open): encoding for CESU-8https://redmine.ruby-lang.org/issues/159312019-06-17T07:20:39Zarton (Akio Tajima)artonx@yahoo.co.jp
<p>RubyとJavaのブリッジを開発していて、文字列の変換処理で困っています。<br>
現象)<br>
JavaのUTF-8は非標準的な形式 (<a href="https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/types.html#modified_utf_8_strings" class="external">https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/types.html#modified_utf_8_strings</a>) なので、Ruby側の文字列をそのままUTF-8にエンコードして与えると、Java側ではサロゲートペアに相当する文字を正しく認識できません。<br>
お願い)<br>
Java固有の拡張UTF-8は一般性が欠けていると考えられますが、ほぼ互換のCESU-8 (<a href="https://www.unicode.org/reports/tr26/tr26-4.html" class="external">https://www.unicode.org/reports/tr26/tr26-4.html</a>) は、IANAの文字セットにも登録されていて通信などでそれなりに利用されているようなので、RubyのEncodingでサポートしていただきたいと思います。</p>
<p>よろしくご検討ください。</p> Ruby master - Misc #15007 (Open): Let all Init_xxx and extension APIs frequently called from init...https://redmine.ruby-lang.org/issues/150072018-08-18T23:17:31Zmethodmissing (Lourens Naudé)lourens@bearmetal.eu
<p>References Github PR <a href="https://github.com/ruby/ruby/pull/1934" class="external">https://github.com/ruby/ruby/pull/1934</a></p>
<a name="Why"></a>
<h3 >Why?<a href="#Why" class="wiki-anchor">¶</a></h3>
<p>An incremental extraction from PR <a href="https://github.com/ruby/ruby/pull/1922" class="external">https://github.com/ruby/ruby/pull/1922</a>, specifically addressing the feedback from Yui Naruse in <a href="https://github.com/ruby/ruby/pull/1922#issuecomment-413796710" class="external">https://github.com/ruby/ruby/pull/1922#issuecomment-413796710</a></p>
<p>The <a href="https://github.com/torvalds/linux/blob/ca04b3cca11acbaf904f707f2d9ca9654d7cc226/include/linux/compiler-gcc.h#L191-L206" class="external">Linux kernel</a>, <a href="https://github.com/php/php-src/blob/2d71a28954a4f20709718ee7cb2b850d334c561c/Zend/zend_portability.h#L220" class="external">PHP 7</a> and other projects use the <code>hot</code> and <code>cold</code> function attributes to help with better code layout.</p>
<p>I noticed Ruby is very much CPU frontend bound (not feeding instructions into the CPU pipelines as fast as it maybe could) and therefore even most micro benchmarks have a high CPI (cycles per instruction) rate. This PR is part of a larger chunk of work I'd like to do around improving CPU frontend throughput and can take a stab at formally writing up those ideas if there's any interest from the community. I don't know.</p>
<a name="Implementation"></a>
<h3 >Implementation<a href="#Implementation" class="wiki-anchor">¶</a></h3>
<p>This PR has an exclusive focus on having the <code>Init_xxx</code> functions for the core classes and those bundled in <code>ext</code> being flagged to be optimized for size as they're called only once at runtime.</p>
<p>The GCC specific <a href="https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Function-Attributes.html" class="external">cold</a> function attribute works in the following way (from GCC docs):</p>
<pre><code>The cold attribute is used to inform the compiler that a function is unlikely executed. The function is optimized for size rather than speed and on many targets it is placed into special subsection of the text section so all cold functions appears close together improving code locality of non-cold parts of program. The paths leading to call of cold functions within code are marked as unlikely by the branch prediction mechanism. It is thus useful to mark functions used to handle unlikely conditions, such as perror, as cold to improve optimization of hot functions that do call marked functions in rare occasions.
When profile feedback is available, via -fprofile-use, hot functions are automatically detected and this attribute is ignored.
</code></pre>
<p>By declaring a function as <code>cold</code> when defined we get the following benefits:</p>
<ul>
<li>No-op on platforms that does not support the attribute</li>
<li>Size optimization of cold functions with a smaller footprint in the instruction cache</li>
<li>Therefore CPU frontend throughput increases due to a lower ratio of instruction cache misses and a lower ITLB overhead - see <a href="https://user-images.githubusercontent.com/379/44204858-4c085100-a14c-11e8-86b8-d87fcb5e4985.png" class="external">original chunky PR</a> VS <a href="https://user-images.githubusercontent.com/379/44204870-4f9bd800-a14c-11e8-9bee-14c8ad8d3a7d.png" class="external">then trunk</a>
</li>
<li>This effect can further be amplified in future work with the <code>hot</code> attribute</li>
</ul>
<a name="Extension-APIs-flagged-as-cold"></a>
<h4 >Extension APIs flagged as cold<a href="#Extension-APIs-flagged-as-cold" class="wiki-anchor">¶</a></h4>
<p>These are and should typically only be called on extension init, and thus safe to optimize for size as well.</p>
<ul>
<li><code>void rb_define_method_id(VALUE, ID, VALUE (*)(ANYARGS), int));</code></li>
<li><code>void rb_undef(VALUE, ID));</code></li>
<li><code>void rb_define_protected_method(VALUE, const char*, VALUE (*)(ANYARGS), int));</code></li>
<li><code>void rb_define_private_method(VALUE, const char*, VALUE (*)(ANYARGS), int));</code></li>
<li><code>void rb_define_singleton_method(VALUE, const char*, VALUE(*)(ANYARGS), int));</code></li>
<li><code>void rb_define_alloc_func(VALUE, rb_alloc_func_t));</code></li>
<li><code>void rb_undef_alloc_func(VALUE));</code></li>
<li><code>VALUE rb_define_class(const char*,VALUE));</code></li>
<li><code>VALUE rb_define_module(const char*));</code></li>
<li><code>VALUE rb_define_class_under(VALUE, const char*, VALUE));</code></li>
<li><code>VALUE rb_define_module_under(VALUE, const char*));</code></li>
<li><code>void rb_define_variable(const char*,VALUE*));</code></li>
<li><code>void rb_define_virtual_variable(const char*,VALUE(*)(ANYARGS),void(*)(ANYARGS)));</code></li>
<li><code>void rb_define_hooked_variable(const char*,VALUE*,VALUE(*)(ANYARGS),void(*)(ANYARGS)));</code></li>
<li><code>void rb_define_readonly_variable(const char*,const VALUE*));</code></li>
<li><code>void rb_define_const(VALUE,const char*,VALUE));</code></li>
<li><code>void rb_define_global_const(const char*,VALUE));</code></li>
<li><code>void rb_define_method(VALUE,const char*,VALUE(*)(ANYARGS),int));</code></li>
<li><code>(void rb_define_module_function(VALUE,const char*,VALUE(*)(ANYARGS),int));</code></li>
<li><code>void rb_define_global_function(const char*,VALUE(*)(ANYARGS),int));</code></li>
<li><code>void rb_undef_method(VALUE,const char*));</code></li>
<li><code>void rb_define_alias(VALUE,const char*,const char*));</code></li>
<li><code>void rb_define_attr(VALUE,const char*,int,int));</code></li>
<li><code>void rb_global_variable(VALUE*));</code></li>
<li><code>void rb_gc_register_mark_object(VALUE));</code></li>
<li><code>void rb_gc_register_address(VALUE*));</code></li>
<li><code>void rb_gc_unregister_address(VALUE*));</code></li>
</ul>
<a name="Text-segment-reductions"></a>
<h4 >Text segment reductions<a href="#Text-segment-reductions" class="wiki-anchor">¶</a></h4>
<p>Small changes (<code>3144</code> bytes reduction of the text segment) because this is incremental groundwork and and initial low risk PR.</p>
<p>this branch:</p>
<pre><code>lourens@CarbonX1:~/src/ruby/ruby$ size ruby
text data bss dec hex filename
3462153 21056 71344 3554553 363cf9 ruby
</code></pre>
<p>trunk:</p>
<pre><code>lourens@CarbonX1:~/src/ruby/trunk$ size ruby
text data bss dec hex filename
3465297 21056 71344 3557697 364941 ruby
</code></pre>
<p>Diffs for individual object files: <a href="https://www.diffchecker.com/T0GVzX1q" class="external">https://www.diffchecker.com/T0GVzX1q</a></p>
<p>Default <code>text.unlikely</code> section where init functions are moved to:</p>
<pre><code>lourens@CarbonX1:~/src/ruby/ruby$ readelf -S vm.o
There are 34 section headers, starting at offset 0x2a04f8:
Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
[ 1] .text PROGBITS 0000000000000000 00000040
000000000001c37f 0000000000000000 AX 0 0 16
[ 2] .rela.text RELA 0000000000000000 00114100
000000000000a7d0 0000000000000018 I 31 1 8
[ 3] .data PROGBITS 0000000000000000 0001c3c0
0000000000000030 0000000000000000 WA 0 0 16
[ 4] .bss NOBITS 0000000000000000 0001c400
00000000000002b0 0000000000000000 WA 0 0 32
[ 5] .rodata.str1.8 PROGBITS 0000000000000000 0001c400
0000000000000d6f 0000000000000001 AMS 0 0 8
[ 6] .text.unlikely PROGBITS 0000000000000000 0001d16f <<<<<<<<<<<<<<<
0000000000001aa9 0000000000000000 AX 0 0 1
</code></pre>
<p>The relocations for <code>vm.o</code>:</p>
<pre><code>lourens@CarbonX1:~/src/ruby/ruby$ ld -M vm.o
--- truncated ---
.text 0x0000000000400120 0x1de2f
*(.text.unlikely .text.*_unlikely .text.unlikely.*)
.text.unlikely
0x0000000000400120 0x1aa9 vm.o
0x000000000040038f rb_define_alloc_func
0x00000000004003bf rb_undef_alloc_func
0x00000000004003c5 Init_Method
0x0000000000400512 Init_vm_eval
0x00000000004007a1 Init_eval_method
0x0000000000400a54 rb_undef
0x0000000000400c1d Init_VM
0x000000000040185f Init_BareVM
0x0000000000401b16 Init_vm_objects
0x0000000000401b61 Init_top_self
*(.text.exit .text.exit.*)
*(.text.startup .text.startup.*)
*(.text.hot .text.hot.*)
*(.text .stub .text.* .gnu.linkonce.t.*)
*fill* 0x0000000000401bc9 0x7
.text 0x0000000000401bd0 0x1c37f vm.o
0x00000000004022f0 rb_f_notimplement
0x0000000000404780 rb_vm_ep_local_ep
0x00000000004047b0 rb_vm_frame_block_handler
0x00000000004047e0 rb_vm_cref_new_toplevel
0x0000000000404870 rb_vm_block_ep_update
0x0000000000404890 ruby_vm_special_exception_copy
0x0000000000406960 rb_ec_stack_overflow
0x00000000004069c0 rb_vm_push_frame
0x0000000000406b20 rb_vm_pop_frame
0x0000000000406b30 rb_error_arity
0x0000000000407180 rb_vm_frame_method_entry
0x00000000004075e0 rb_vm_rewrite_cref
0x00000000004076f0 rb_simple_iseq_p
0x0000000000407700 rb_vm_opt_struct_aref
0x0000000000407730 rb_vm_opt_struct_aset
0x0000000000407750 rb_clear_constant_cache
--- truncated ---
</code></pre>
<p>I also dabbled with the idea of an <code>INITFUNC</code> macro that also places the <code>Init_xxx</code> functions into a <code>text.init</code> section as the <a href="https://linuxgazette.net/157/amurray.html" class="external">kernel does</a> for a possible future optimization of stripping out ELF sections for setup / init specific functions. I don't think that makes sense for now and possibly only interesting for mruby or embedded.</p>
<a name="Possible-next-units-of-work"></a>
<h3 >Possible next units of work<a href="#Possible-next-units-of-work" class="wiki-anchor">¶</a></h3>
<a name="Cold-code-specific"></a>
<h4 >Cold code specific<a href="#Cold-code-specific" class="wiki-anchor">¶</a></h4>
<ul>
<li>Incrementally PR corner case error handling functions such as <code>rb_bug</code> from <a href="https://github.com/ruby/ruby/pull/1922" class="external">https://github.com/ruby/ruby/pull/1922</a>
</li>
<li>Ditto for generic error handling functions (<code>rb_raise</code> and friends) from <a href="https://github.com/ruby/ruby/pull/1922" class="external">https://github.com/ruby/ruby/pull/1922</a>
</li>
<li>Class specific error handling functions (load errors, encoding errors in the IO module, sys errors etc.) from <a href="https://github.com/ruby/ruby/pull/1922" class="external">https://github.com/ruby/ruby/pull/1922</a>
</li>
<li>GCC 5+ also supports <code>cold</code> <a href="https://gcc.gnu.org/onlinedocs/gcc/Label-Attributes.html" class="external">labels</a> , which I took a stab with in the bloated <a href="https://github.com/ruby/ruby/pull/1922" class="external">https://github.com/ruby/ruby/pull/1922</a>
</li>
</ul>
<a name="TLB-translation-lookaside-buffer-specific"></a>
<h4 >TLB (translation lookaside buffer) specific<a href="#TLB-translation-lookaside-buffer-specific" class="wiki-anchor">¶</a></h4>
<ul>
<li>Further ITLB overhead investigation</li>
<li>Ruby binaries built with O3 and debug symbols come in at just short of 18MB, or roughly 9 hugepages on linux. PHP core developers were able to squeeze a few % by remapping code to hugepages on supported systems - <a href="http://developers-club.com/posts/270685/" class="external">http://developers-club.com/posts/270685/</a> . Implementation <a href="https://github.com/php/php-src/blob/fb0389b1010de5a6459bcf286409423f69e74aaf/ext/opcache/ZendAccelerator.c#L2645-L2750" class="external">here</a>
</li>
</ul>
<a name="Bytecode-specific"></a>
<h4 >Bytecode specific<a href="#Bytecode-specific" class="wiki-anchor">¶</a></h4>
<ul>
<li>The <a href="https://software.intel.com/en-us/vtune-amplifier-help-task-api" class="external">Intel Tracing Task API</a> is very well suited for the instruction sequences YARV generates and to infer better per instruction CPU utilization and identify any stalls (frontend, backend, branches etc.) to drive further work.</li>
</ul> Ruby master - Misc #10628 (Open): Peformance of URI modulehttps://redmine.ruby-lang.org/issues/106282014-12-21T14:43:44Ztgxworld (Guo Xiang Tan)gxtan1990@gmail.com
<p>Please view attached screenshot or go to <a href="https://railsbench.herokuapp.com/tgxworld/ruby?utf8=%E2%9C%93&result_types%5B%5D=app_uri&commit=Submit" class="external">the following link</a> to see benchmark graph over time.</p>
<p>It got slower after this <a href="https://github.com/ruby/ruby/commit/bb83f32dc3e0424d25fa4e55d8ff32b061320e41" class="external">commit</a>.</p>
<p>Hope this helps.</p> Ruby master - Feature #9020 (Assigned): Net::HTTPResponse predicate/query methodshttps://redmine.ruby-lang.org/issues/90202013-10-14T20:43:15Ztimcraft (Tim Craft)
<a name="SUMMARY"></a>
<h1 >SUMMARY<a href="#SUMMARY" class="wiki-anchor">¶</a></h1>
<p>I would like to propose adding predicate/query methods to Net::HTTPResponse for testing the status/type of response. For example:</p>
<pre><code>response.ok?
response.not_found?
response.client_error?
response.server_error?
</code></pre>
<a name="BACKGROUND"></a>
<h1 >BACKGROUND<a href="#BACKGROUND" class="wiki-anchor">¶</a></h1>
<p>The approach I've most commonly used/encountered for testing the status of a response is to compare with an integer, for example:</p>
<pre><code>response.code.to_i == 200
</code></pre>
<p>Subjectively I could say this kind of code is awkward/tedious to type, and not very "intention revealing". More practically/objectively it's a potential source of error. By mistyping the "magic number" it's possible for this expression to "silently fail" and test the wrong status. That would be an easy thing to spot in these examples, but much more difficult to track down within a typical codebase.</p>
<p>Another approach would be to test the type/class of the response object, for example:</p>
<pre><code>Net::HTTPOK === response
</code></pre>
<p>Subjectively I would say this doesn't feel very Ruby-ish. More practically/objectively it tightly couples the caller to the implementation details of Net::HTTP, making it difficult to stub or swap in a different library.</p>
<a name="PROPOSAL"></a>
<h1 >PROPOSAL<a href="#PROPOSAL" class="wiki-anchor">¶</a></h1>
<p>I would like to propose adding predicate/query methods to Net::HTTPResponse in order to encapsulate the implementation details of testing for different statuses and to provide a more abstract interface to the caller. For example:</p>
<pre><code>response.ok?
response.not_found?
</code></pre>
<p>This is more concise/readable. In most cases it would be easier and "less fiddly" to type out than the existing approaches presented above.</p>
<p>Compared to testing with integers it is one method call instead of three (I'm considering that as better from a readability perspective, not a performance perspective), and it eliminates the "failing silently" issue.</p>
<p>Compared to testing the type/class of the response object it doesn't couple the caller to implementation details of Net::HTTP, so it would be easier to stub or swap-in another library that provides the same interface.</p>
<p>Overall it feels much simpler and much more Ruby-ish.</p>
<p>In addition I would propose adding a few extra methods to test for ranges of statuses, for example:</p>
<pre><code>response.client_error?
response.server_error?
</code></pre>
<p>Similar benefits/rationale.</p>
<a name="IMPLEMENTATION"></a>
<h1 >IMPLEMENTATION<a href="#IMPLEMENTATION" class="wiki-anchor">¶</a></h1>
<p>I have already been using methods like this in some gems, and I have created a "proof of concept" implementation which monkey-patches Net::HTTP to test the idea out. Available here:</p>
<pre><code>http://rubygems.org/gems/net-http-predicates
https://github.com/timcraft/net-http-predicates
</code></pre>
<p>I can think of various different ways to implement a patch, so if this feature would be accepted into ruby-trunk I would welcome suggestions/guidance on a preferred implementation.</p>
<p>These changes would be backwards compatible and straightforward to provide as a backport, either in the backports library/gem or as a standalone gem.</p>
<a name="DISCUSSION"></a>
<h1 >DISCUSSION<a href="#DISCUSSION" class="wiki-anchor">¶</a></h1>
<p>Before discussing how to implement this patch I would like to get people's thoughts on the idea/proposal, and some indication of whether this could be accepted into ruby-trunk (or not). If it would be accepted I'm happy to write/submit the patch itself.</p> Ruby master - Bug #7892 (Open): MIME encoding bug of NKF.nkfhttps://redmine.ruby-lang.org/issues/78922013-02-20T16:01:52Zmrkn (Kenta Murata)muraken@gmail.com
<p>NKF の MIME encoding の結果が 1.8 と 1.9/2.0 で異なってます。</p>
<a name="18-の場合"></a>
<h1 >1.8 の場合<a href="#18-の場合" class="wiki-anchor">¶</a></h1>
<p>$ /usr/bin/ruby -rnkf -ve "puts NKF.nkf('-jW -M --cp932', '「あああああああああああ by ああああああああああ」のレシピ')"<br>
ruby 1.8.7 (2012-02-08 patchlevel 358) [universal-darwin11.0]<br>
=?ISO-2022-JP?B?GyRCIVYkIiQiJCIkIiQiJCIkIiQiJCIkIiQiGyhC?= by<br>
=?ISO-2022-JP?B?GyRCJCIkIiQiJCIkIiQiJCIkIiQiJCIhVyROJWwlNyVUGyhC?=</p>
<a name="193-p385-の場合"></a>
<h1 >1.9.3-p385 の場合<a href="#193-p385-の場合" class="wiki-anchor">¶</a></h1>
<p>$ ruby -rnkf -ve "puts NKF.nkf('-jW -M --cp932', '「あああああああああああ by ああああああああああ」のレシピ')"<br>
ruby 1.9.3p385 (2013-02-06 revision 39114) [x86_64-darwin11.4.2]<br>
=?ISO-2022-JP?B?GyRCIVYkIiQiJCIkIiQiJCIkIiQiJCIkIiQiGyhC?= by<br>
=?US-ASCII?Q??=<br>
=?ISO-2022-JP?B?GyRCJCIkIiQiJCIkIiQiJCIkIiQiJCIhVyROJWwlNyVUGyhC?=</p>
<a name="200-rc2-の場合"></a>
<h1 >2.0.0-rc2 の場合<a href="#200-rc2-の場合" class="wiki-anchor">¶</a></h1>
<p>$ RBENV_VERSION=2.0.0-rc2 rbenv exec ruby -rnkf -ve "puts NKF.nkf('-jW -M --cp932', '「あああああああああああ by ああああああああああ」のレシピ')"<br>
ruby 2.0.0dev (2013-02-08 trunk 39161) [x86_64-darwin11.4.2]<br>
=?ISO-2022-JP?B?GyRCIVYkIiQiJCIkIiQiJCIkIiQiJCIkIiQiGyhC?= by<br>
=?US-ASCII?Q??=<br>
=?ISO-2022-JP?B?GyRCJCIkIiQiJCIkIiQiJCIkIiQiJCIhVyROJWwlNyVUGyhC?=</p> Ruby master - Feature #6265 (Assigned): Remove 'useless' 'concatenation' syntaxhttps://redmine.ruby-lang.org/issues/62652012-04-06T21:53:53Zrosenfeld (Rodrigo Rosenfeld Rosas)rr.rosas@gmail.com
<p>What is wrong with this code:</p>
<p>some_method 'argument1', 'argument2' 'argument3'</p>
<p>Yes, the missing colon, but it is not always easy to notice that...</p>
<p>What is this ('concatenation' 'syntax') useful for?</p>
<p>Why writing ('some ' 'concatenation') instead of 'some concatenation'?</p>
<p>A missing colon between string arguments can lead to some bugs that may be hard to find, specially if the arguments are optional.</p>
<p>And I can't see any useful case where this allowed syntax for concatenation would help.</p> Ruby master - Feature #5764 (Assigned): Net::HTTP should assume HTTP/0.9 on unexpected responseshttps://redmine.ruby-lang.org/issues/57642011-12-15T06:01:34Zmstyer (Mike Styer)michael@styer.net
<p>Currently Net::HTTP.read_status_line throws Net::HTTPBadResponse if the status line does not conform to HTTP/1.1 specifications.</p>
<p>But in cases when the web server implements a request size limit, it may not read HTTP/1.1 trailer after the request URI and may send back an HTTP/0.9 response.</p>
<p>Nginx does this for 414 Request-URI Too Large responses:</p>
<p><a href="http://lxr.evanmiller.org/http/source/http/ngx_http_header_filter_module.c#L95" class="external">http://lxr.evanmiller.org/http/source/http/ngx_http_header_filter_module.c#L95</a><br>
<a href="http://forum.nginx.org/read.php?2,52862,52862" class="external">http://forum.nginx.org/read.php?2,52862,52862</a></p>
<p>Perl's Net::HTTP provides a "laxed" option to read_response_headers() to assume HTTP/0.9 if it can't find an HTTP/1.1 status line. Ruby should provide a similar option.</p> Ruby master - Feature #5461 (Assigned): Add pipelining to Net::HTTPhttps://redmine.ruby-lang.org/issues/54612011-10-19T07:37:41Zdrbrain (Eric Hodel)drbrain@segment7.net
<p>=begin<br>
The attached patch adds HTTP/1.1 pipelining support to Net::HTTP.</p>
<p>Pipelining is only performed on HTTP/1.1 servers. Net::HTTP will check if the server supports pipelining by using the first request in the list. The user can override this via setting (({http.pipelining = true})).</p>
<p>If a server does not support pipelining or there is an error during pipelining an error will be raised that contains the requests that not have been delivered yet and the responses that have been received.</p>
<p>The patch includes documentation explaining the fine details.</p>
<p>Example:</p>
<p>requests = []<br>
requests << Net::HTTP::Get.new('/images/bug.png')<br>
requests << Net::HTTP::Get.new('/images/date.png')<br>
requests << Net::HTTP::Get.new('/images/find.png')</p>
<p>http = Net::HTTP.new 'localhost'<br>
http.start do<br>
http.pipeline requests do |req, res|<br>
open File.basename(req.path), 'wb' do |io|<br>
io.write res.body<br>
end<br>
end<br>
end</p>
<p>Implementation notes:</p>
<ul>
<li>The current Net::HTTP tests make it very difficult to test bad behavior by servers. In test/net/http/utils.rb I introduced a method to replace Net::BufferedIO with a subclass that can behave incorrectly.</li>
<li>Net::HTTP#pipeline does not fall back to sending requests one-by-one for HTTP/1.1 servers. I think this is acceptable as the user can use existing Net::HTTP code to send requests one-by-one.</li>
</ul>
<p>=end</p>