Ruby Issue Tracking System: Issues
https://redmine.ruby-lang.org/
https://redmine.ruby-lang.org/favicon.ico?1711330511
2020-12-25T09:32:50Z
Ruby Issue Tracking System
Redmine
Ruby master - Feature #17468 (Closed): Deprecate RUBY_DEVEL
https://redmine.ruby-lang.org/issues/17468
2020-12-25T09:32:50Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Some configuration of Ruby use RUBY_DEVEL, which depends PATCH_LEVEL.<br>
But depending PATCH_LEVEL causes issues which will become revealed on the final release.<br>
Though we release some previews and RCs, they don't contributes the quality around RUBY_DEVEL.</p>
<p>Therefore to ensure CI tests the quality of the final release, we need to deprecate RUBY_DEVEL.</p>
Ruby master - Misc #17376 (Assigned): Reduce number of GitHub Actions
https://redmine.ruby-lang.org/issues/17376
2020-12-08T09:45:17Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>At this time we have 127 checks for GitHub commits, but unfortunately GitHub UI only shows 100 checks.<br>
It sometimes makes we don't see a failed check.<br>
Could you reduce number of GitHub actions at least less than 100?</p>
Ruby master - Feature #16275 (Closed): Revert `.:` syntax
https://redmine.ruby-lang.org/issues/16275
2019-10-23T15:26:13Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p><code>obj.:method</code> is introduced at r66667 by <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Proposal: Shorthand operator for Object#method (Open)" href="https://redmine.ruby-lang.org/issues/12125">#12125</a> and <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Syntax sugar for method reference (Closed)" href="https://redmine.ruby-lang.org/issues/13581">#13581</a>.<br>
It encourages the functional programming style in Ruby.</p>
<p>But this shorthand syntax is just for methods of <code>self</code> without arguments.<br>
It causes another feature requests like <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Proposal: Shorthand operator for "#instance_method" (Open)" href="https://redmine.ruby-lang.org/issues/16273">#16273</a> (and lambda compositions like <a class="issue tracker-1 status-7 priority-4 priority-default closed" title="Bug: Refactor Proc#>> and #<< (Feedback)" href="https://redmine.ruby-lang.org/issues/15428">#15428</a>).</p>
<p>Such features will introduce a new view of Ruby but no one illustrates the whole picture yet.<br>
I worried about such patch work may cause a conflict with future expansion of functional programing style or a just a garbage feature.</p>
<p><code>.:</code> syntax is introduced in 2.7.0 preview1, not released in production yet.<br>
How about reverting at this time and re-introduce with a big picture.</p>
Ruby master - Feature #16131 (Closed): Remove $SAFE, taint and trust
https://redmine.ruby-lang.org/issues/16131
2019-08-29T07:14:34Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Ruby had Taint checking which is originally introduced in Perl.<br>
<a href="https://en.wikipedia.org/wiki/Taint_checking" class="external">https://en.wikipedia.org/wiki/Taint_checking</a></p>
<p>It was intended to provide a useful tool for handle objects which are come from outside.<br>
Input data is set as tainted by default and call untaint if you checked or filtered the value.<br>
Some people used this feature in the age of CGI.</p>
<p>But these days, no one use the mechanism and input libraries usually doesn't support it.<br>
For example rack, as following shows its input is not tainted and the mechanism is unusable.</p>
<pre><code>% cat foo.ru
run ->(env) do
['200', {'Content-Type' => 'text/plain'}, ["Is QUERY_STRING tainted?: #{env["QUERY_STRING"].tainted?}"]]
end
% rackup foo.ru
[51724] Puma starting in cluster mode...
[51724] * Version 3.12.1 (ruby 2.6.3-p62), codename: Llamas in Pajamas
[51724] * Min threads: 3, max threads: 3
[51724] * Environment: development
[51724] * Process workers: 1
[51724] * Preloading application
[51724] * Listening on tcp://localhost:9292
[51724] Use Ctrl-C to stop
[51737] + Gemfile in context: /Users/naruse/work/td-cdp-api/Gemfile
[51724] - Worker 0 (pid: 51737) booted, phase: 0
</code></pre>
<pre><code>% curl http://localhost:9292/\?foo=1
Is QUERY_STRING tainted?: false
</code></pre>
<p>Therefore I think Taint checking mechanism is unusable on the current Ruby ecosystem.</p>
<p>On the other hand we experienced multiple vulnerability around $SAFE and taint mechanism.<br>
<a href="https://cse.google.com/cse?q=taint&cx=008288045305770251182%3Afvruzsaknew&ie=UTF-8" class="external">https://cse.google.com/cse?q=taint&cx=008288045305770251182%3Afvruzsaknew&ie=UTF-8</a><br>
The cost of maintaining it is expensive.</p>
<p>In conclusion, I think the taint mechanism is too expensive to maintain for the merit of it.<br>
I suggest to remove it.</p>
Ruby master - Bug #15992 (Closed): An exception breaks monitor state and cause deadlock
https://redmine.ruby-lang.org/issues/15992
2019-07-10T06:55:40Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>lib/monitor.rb provides Monitor.<br>
But its state handling is weak for interrupts caused by Thread.kill for example timeout libraries.</p>
<p>Timeout exception may happen everywhere. If it raised when the thread is executing</p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="k">def</span> <span class="nf">mon_exit</span>
<span class="n">mon_check_owner</span>
<span class="vi">@mon_count</span> <span class="o">-=</span><span class="mi">1</span>
<span class="k">if</span> <span class="vi">@mon_count</span> <span class="o">==</span> <span class="mi">0</span>
<span class="vi">@mon_owner</span> <span class="o">=</span> <span class="kp">nil</span>
<span class="c1"># HERE!!!</span>
<span class="vi">@mon_mutex</span><span class="p">.</span><span class="nf">unlock</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
<p>It breaks the state of the monitor and it causes deadlock.</p>
Ruby master - Feature #15958 (Closed): Time#inspect with frac
https://redmine.ruby-lang.org/issues/15958
2019-06-25T13:12:18Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>When Matz changed Time#to_s format in 2006, frac part of Time string is dropped because it is considered not useful.<br>
<a href="https://blade.ruby-lang.org/ruby-dev/29440">[ruby-dev:29440]</a></p>
<p>But recently we encounters some troubles the comparison of Time objects whose frac parts are different.<br>
<a href="https://twitter.com/__gfx__/status/1143056072422219776" class="external">https://twitter.com/__gfx__/status/1143056072422219776</a></p>
<p>For example a is original object and b is once stored in RDB (and dropped the frac part).<br>
Or there're multiple time objects which are generated in a single HTTP request.<br>
Of course they are different and assert_equal will be failed but inspect doesn't show the frac part.</p>
Ruby master - Bug #15398 (Closed): TestThread#test_signal_at_join fails on FreeBSD
https://redmine.ruby-lang.org/issues/15398
2018-12-11T09:10:02Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Since r64538 or r64539, TestThread#test_signal_at_join is failing on our FreeBSD CI.</p>
<pre><code>+ <n>) Error:
+TestThread#test_signal_at_join:
+Timeout::Error: execution of assert_separately expired timeout (120 sec)
+pid 80587 killed by SIGTERM (signal 15)
+| BAhvOhRTaWduYWxFeGNlcHRpb24KOgltZXNnIgxTSUdURVJNOgdidFsMSSIU
+| LTozNzppbiBgd3JpdGUnBjoGRUZJIhMtOjM3OmluIGBwdXRzJwY7CEZJIikt
+| OjM3OmluIGBibG9jayAoMiBsZXZlbHMpIGluIDxtYWluPicGOwhGSSIULToz
+| NDppbiBgdGltZXMnBjsIRkkiHi06MzQ6aW4gYGJsb2NrIGluIDxtYWluPicG
+| OwhGSSIULToxMDppbiBgcG9wZW4nBjsIRkkiFS06MTA6aW4gYDxtYWluPicG
+| OwhGOgpzaWdub2kUOgpjYXVzZTA6EWJ0X2xvY2F0aW9uc0AH
+| assertions=122
+|
</code></pre>
<p><a href="https://rubyci.org/logs/rubyci.s3.amazonaws.com/freebsd11zfs/ruby-trunk/log/20180826T003001Z.diff.html.gz" class="external">https://rubyci.org/logs/rubyci.s3.amazonaws.com/freebsd11zfs/ruby-trunk/log/20180826T003001Z.diff.html.gz</a></p>
<p>Note that this doesn't happen on my private FreeBSD box.</p>
Ruby master - Bug #15164 (Closed): mkmf doesn't work with miniruby on Windows
https://redmine.ruby-lang.org/issues/15164
2018-09-26T15:29:46Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>After r59449 <code>nmake</code> failes to run when making ripper and so on.<br>
It because miniruby sets UTF-8 encoding even though the content is CP932 (on Japanese Windows).<br>
It should set ASCII-8BIT and handle as it is.</p>
<pre><code>ripper:
" Could not be configured. It will not be installed."
" C:/ruby-trunk/lib/mkmf.rb:1560: invalid byte sequence in UTF-8"
" Check ext/ripper/mkmf.log for more details."
*** Fix the problems, then remove these directories and try again if you want.
</code></pre>
<p>Note that recent Windows sets PATH to paths under %APPDATA% for example<br>
C:\Users<UserNameInANSI>\AppData\Local\Microsoft\WindowsApps</p>
Ruby master - Misc #14861 (Closed): DevelopersMeeting20180718Japan
https://redmine.ruby-lang.org/issues/14861
2018-06-21T05:58:04Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Date: 2018/07/18 (Thu)<br>
Time: 14:00-18:00 (JST)<br>
Place: MoneyForward HQ (beware: new location) (Tokyo, Japan)<br>
Sign-up: <a href="https://ruby.connpass.com/event/92314/" class="external">https://ruby.connpass.com/event/92314/</a><br>
log: <a href="https://docs.google.com/document/d/1_cKh0LJd18y5CH1MfM6WC1fqh2rpHHZntrBCLTbwRSE/edit" class="external">https://docs.google.com/document/d/1_cKh0LJd18y5CH1MfM6WC1fqh2rpHHZntrBCLTbwRSE/edit</a><br>
pub: <a href="https://docs.google.com/document/d/1_cKh0LJd18y5CH1MfM6WC1fqh2rpHHZntrBCLTbwRSE/pub" class="external">https://docs.google.com/document/d/1_cKh0LJd18y5CH1MfM6WC1fqh2rpHHZntrBCLTbwRSE/pub</a></p>
<p>Please comment your favorite ticket numbers you want to ask to discuss with your <em>SHORT</em> comment or summary.<br>
(your summary/comment will help us because we don't need to read all of ticket comments)</p>
<p><em>DO NOT</em> discuss then on this ticket, please.</p>
<p>Past meetings: <a href="https://bugs.ruby-lang.org/projects/ruby/wiki#Developer-Meetings" class="external">https://bugs.ruby-lang.org/projects/ruby/wiki#Developer-Meetings</a></p>
<a name="NOTE"></a>
<h1 >NOTE<a href="#NOTE" class="wiki-anchor">¶</a></h1>
<p>Dev meeting <em>IS NOT</em> a decision making place. All decisions should be done at the bug tracker.<br>
Dev meeting is a place we can ask Matz, nobu, nurse and other developers directly.<br>
Matz is a very busy person. Take this opportunity to ask him. If you can not attend, other attendees can ask instead of you (if attendees can understand your issue).<br>
We will write a log about discussion to a file or to each ticket in English.<br>
All activities are best-effort (keep in mind that most of us are volunteer developers).<br>
The date, time and place is scheduled according to when/where we can reserve Matz's time.</p>
<a name="Agenda"></a>
<h1 >Agenda<a href="#Agenda" class="wiki-anchor">¶</a></h1>
<a name="Next-dev-meeting"></a>
<h2 >Next dev-meeting<a href="#Next-dev-meeting" class="wiki-anchor">¶</a></h2>
<a name="About-26-timeframe"></a>
<h2 >About 2.6 timeframe<a href="#About-26-timeframe" class="wiki-anchor">¶</a></h2>
<a name="Carry-over-from-previous-meetings"></a>
<h2 >Carry-over from previous meeting(s)<a href="#Carry-over-from-previous-meetings" class="wiki-anchor">¶</a></h2>
<ul>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Comparable#clamp with a range (Closed)" href="https://redmine.ruby-lang.org/issues/14784">#14784</a>] One-sided Comparable#clamp (with endless/startless ranges) (zverok)
<ul>
<li>more reasonable version of Object#enumerate proposed for the previous meeting.</li>
</ul>
</li>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: [PATCH] implement Timeout in VM (Open)" href="https://redmine.ruby-lang.org/issues/14859">#14859</a>] Timeout in VM (normalperson)
<ul>
<li>Still needs some work, mainly wondering if the idea of moving this part of stdlib into core VM is acceptable or not. No semantic changes except speed improvement.</li>
</ul>
</li>
</ul>
<a name="From-Attendees"></a>
<h2 >From Attendees<a href="#From-Attendees" class="wiki-anchor">¶</a></h2>
<p>(will be edited later)<br>
(if you have a write access permission, please list directly)</p>
<ul>
<li>[Bug <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Array#delete_if does not use #delete (Closed)" href="https://redmine.ruby-lang.org/issues/14887">#14887</a>] Array#delete_if does not use #delete (shyouhei)
<ul>
<li>Is it by design, or a bug?</li>
</ul>
</li>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Readline: expose rl_completion_quote_character variable (Closed)" href="https://redmine.ruby-lang.org/issues/13050">#13050</a>] Readline: expose rl_completion_quote_character variable (nobu)</li>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Add official API for setting timezone on Time (Closed)" href="https://redmine.ruby-lang.org/issues/14850">#14850</a>] Add official API for setting timezone on Time (nobu)</li>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Proposal to add Hash#=== (Open)" href="https://redmine.ruby-lang.org/issues/14869">#14869</a>] Proposal to add Hash#=== (nobu)</li>
<li>[Feature <a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Calculate age in Date class (Rejected)" href="https://redmine.ruby-lang.org/issues/14877">#14877</a>] Calculate age in Date class (nobu)</li>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Add Range#subrange? (Closed)" href="https://redmine.ruby-lang.org/issues/14473">#14473</a>] Add Range#subrange? (tarui)</li>
<li>[Bug <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: Enumerator::Lazy creates unnecessary Array objects. (Closed)" href="https://redmine.ruby-lang.org/issues/14908">#14908</a>] Enumerator::Lazy creates unnecessary Array objects. (nobu)</li>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Introduce pattern matching syntax (Closed)" href="https://redmine.ruby-lang.org/issues/14912">#14912</a>] Introduce pattern matching syntax (mame)
<ul>
<li>pattern matching</li>
</ul>
</li>
<li>opt_to_s (nobu)
<ul>
<li>see below for his patch</li>
</ul>
</li>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Add String#byteslice! (Open)" href="https://redmine.ruby-lang.org/issues/13626">#13626</a>] Add String#byteslice! (aycabta)</li>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Add display width method to String for CLI (Open)" href="https://redmine.ruby-lang.org/issues/14618">#14618</a>] Add display width method to String for CLI (aycabta)</li>
<li>[Misc <a class="issue tracker-5 status-2 priority-4 priority-default" title="Misc: Add RDoc documents to tar ball (Assigned)" href="https://redmine.ruby-lang.org/issues/14917">#14917</a>] Add RDoc documents to tar ball (aycabta)</li>
<li>[Feature <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: Use Reline for fallback of ext/readline (Closed)" href="https://redmine.ruby-lang.org/issues/14918">#14918</a>] Use Reline for fallback of ext/readline (aycabta)</li>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Add String#byteinsert (Open)" href="https://redmine.ruby-lang.org/issues/14919">#14919</a>] Add String#byteinsert (aycabta)</li>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Proposal to add Array#=== (Open)" href="https://redmine.ruby-lang.org/issues/14916">#14916</a>] Proposal to add Array#=== (aycabta)</li>
</ul>
<a name="From-non-attendees"></a>
<h2 >From non-attendees<a href="#From-non-attendees" class="wiki-anchor">¶</a></h2>
<p>(will be edited later)<br>
(if you have a write access, please list directly)</p>
<ul>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: ArgumentErrorが発生した時メソッドのプロトタイプをメッセージに含む (Open)" href="https://redmine.ruby-lang.org/issues/14111">#14111</a>] ArgumentErrorが発生した時メソッドのプロトタイプをメッセージに含む (esjee)
<ul>
<li>Suggestion: include method parameters when generating an ArgumentsError message. Nobu's patch in the issue provides functionality to make writing a ruby gem to provide this functionality possible.</li>
</ul>
</li>
<li>[Bug <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Add command line argument to deactivate JIT (Closed)" href="https://redmine.ruby-lang.org/issues/14878">#14878</a>] Add command line argument to deactivate JIT (k0kubun)
<ul>
<li>Please discuss the necessity of the flag and its name in the proposal.</li>
</ul>
</li>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Implement String #blank? #present? and improve #strip and family to handle unicode (Open)" href="https://redmine.ruby-lang.org/issues/12306">#12306</a>] Implement String #blank? #present? and improve #strip and family to handle unicode (sam.saffron)</li>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Extend case to match several values at once (Closed)" href="https://redmine.ruby-lang.org/issues/14913">#14913</a>] Extend case to match several values at once (zverok)
<ul>
<li>some steps towards better pattern matching</li>
</ul>
</li>
<li>[Feature <a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Add BasicObject#instance_exec_with_block (Rejected)" href="https://redmine.ruby-lang.org/issues/14914">#14914</a>] Add BasicObject#instance_exec_with_block (jeremyevans0)</li>
<li>[Feature <a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Deprecate String#crypt (Rejected)" href="https://redmine.ruby-lang.org/issues/14915">#14915</a>] Deprecate String#crypt, move implementation to string/crypt (jeremyevans0)</li>
</ul>
Ruby master - Misc #14770 (Open): [META] DevelopersMeeting
https://redmine.ruby-lang.org/issues/14770
2018-05-17T12:28:50Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>A meta ticket to organize DevelopersMeeting tickets.<br>
<a href="https://bugs.ruby-lang.org/projects/ruby/wiki#Developer-Meetings" class="external">https://bugs.ruby-lang.org/projects/ruby/wiki#Developer-Meetings</a></p>
Ruby master - Bug #14208 (Closed): raise error if value contains CR/LF in iniheader of initialize...
https://redmine.ruby-lang.org/issues/14208
2017-12-20T12:04:47Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>like r59693, initialize_http_header also should raise error.</p>
<p><a href="https://twitter.com/DouweM/status/943441930142220289" class="external">https://twitter.com/DouweM/status/943441930142220289</a></p>
Ruby master - Bug #14186 (Closed): cannot build ruby with tarball on noruby environment
https://redmine.ruby-lang.org/issues/14186
2017-12-14T16:35:37Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>r61055 breaks build</p>
Ruby master - Bug #13945 (Closed): Backport r60024
https://redmine.ruby-lang.org/issues/13945
2017-09-27T21:48:05Z
naruse (Yui NARUSE)
naruse@airemix.jp
<pre><code>vm.c: fetch retval iff necessary
* vm.c (rb_vm_make_jump_tag_but_local_jump): get rid of fetching
retval when it is not used. it is necessary for local jump
state only.
</code></pre>
<p>This caused SEGV if an application which embeds Ruby and uses <code>rb_load_protect</code>.<br>
<a href="https://github.com/vim/vim/pull/2147" class="external">https://github.com/vim/vim/pull/2147</a></p>
Ruby master - Feature #13881 (Open): Use getcontext/setcontext on OS X
https://redmine.ruby-lang.org/issues/13881
2017-09-08T09:10:50Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>getcontext/setcontext is first appeared on OS X 10.5 but deprecated on 10.6.<br>
It seems because POSIX removed them from recent specs.</p>
<p>IEEE Std 1003.1, 2004 Edition says makecontext's use of function declarators with empty parentheses<br>
is an obsolescent feature.<br>
<a href="http://pubs.opengroup.org/onlinepubs/009695399/functions/makecontext.html" class="external">http://pubs.opengroup.org/onlinepubs/009695399/functions/makecontext.html</a></p>
<p>Then POSIX.1-2008 removed those functions.</p>
<p>But OS X 10.13 still has them maybe because some essential applications uses them for co-routines.<br>
Therefore we can use them for performance.</p>
<pre><code>diff --git a/configure.in b/configure.in
index 08e109317f..3e75eb3cf2 100644
--- a/configure.in
+++ b/configure.in
@@ -1142,8 +1142,6 @@ AS_CASE(["$target_os"],
ac_cv_header_syscall_h=no
])
AS_IF([test $macosx_10_5 = yes], [
- ac_cv_func_getcontext=no
- ac_cv_func_setcontext=no
], [
AC_DEFINE(BROKEN_SETREUID, 1)
AC_DEFINE(BROKEN_SETREGID, 1)
diff --git a/cont.c b/cont.c
index c86095775c..f94883ef02 100644
--- a/cont.c
+++ b/cont.c
@@ -65,7 +65,15 @@
#ifndef _WIN32
#include <unistd.h>
#include <sys/mman.h>
-#include <ucontext.h>
+# ifdef __APPLE__
+/* avoid deprecated maks on ucontext.h */
+int getcontext(ucontext_t *);
+void makecontext(ucontext_t *, void (*)(), int, ...);
+int setcontext(const ucontext_t *);
+int swapcontext(ucontext_t * __restrict, const ucontext_t * __restrict);
+# else
+# include <ucontext.h>
+# endif
#endif
#define RB_PAGE_SIZE (pagesize)
#define RB_PAGE_MASK (~(RB_PAGE_SIZE - 1))
</code></pre>
Ruby master - Feature #13873 (Closed): Optimize Dir.glob with FNM_EXTGLOB
https://redmine.ruby-lang.org/issues/13873
2017-09-05T17:51:27Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Rails often queries Dir.glob with complex pattern.</p>
<p>On accessing Rails site, it calls glob as follows on querying templates.<br>
Note that Dir[] insists File::FNM_EXTGLOB.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="k">def</span> <span class="nf">find_template_paths</span><span class="p">(</span><span class="n">query</span><span class="p">)</span>
<span class="no">Dir</span><span class="p">[</span><span class="n">query</span><span class="p">].</span><span class="nf">uniq</span><span class="p">.</span><span class="nf">reject</span> <span class="k">do</span> <span class="o">|</span><span class="n">filename</span><span class="o">|</span>
<span class="no">File</span><span class="p">.</span><span class="nf">directory?</span><span class="p">(</span><span class="n">filename</span><span class="p">)</span> <span class="o">||</span>
<span class="c1"># deals with case-insensitive file systems.</span>
<span class="o">!</span><span class="no">File</span><span class="p">.</span><span class="nf">fnmatch</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="n">filename</span><span class="p">,</span> <span class="no">File</span><span class="o">::</span><span class="no">FNM_EXTGLOB</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
<p><a href="https://github.com/rails/rails/blob/6f1c18308ebffc97d51440cdeed7be71de58f26a/actionview/lib/action_view/template/resolver.rb#L247" class="external">https://github.com/rails/rails/blob/6f1c18308ebffc97d51440cdeed7be71de58f26a/actionview/lib/action_view/template/resolver.rb#L247</a></p>
<p>For example on accessing index page of Post Model (/posts), it calls following glob:<br>
Dir["/home/naruse/foo/app/views/posts/index{.en,}{.html,.text,.js,.css,.ics,.csv,.vcf,.png,.jpeg,.gif,.bmp,.tiff,.svg,.mpeg,.xml,.rss,.atom,.yaml,.multipart_form,.url_encoded_form,.json,.pdf,.zip,.gzip,}{}{.raw,.erb,.html,.builder,.ruby,.coffee,.jbuilder,}"]'</p>
<p>It calls many stat as follows:</p>
<pre><code>% sudo strace ./miniruby58882 -e'Dir["/home/naruse/foo/app/views/posts/index{.en,}{.html,.text,.js,.css,.ics,.csv,.vcf,.png,.jpeg,.gif,.bmp,.tiff,.svg,.mpeg,.xml,.rss,.atom,.yaml,.multipart_form,.url_encoded_form,.json,.pdf,.zip,.gzip,}{}{.raw,.erb,.html,.builder,.ruby,.coffee,.jbuilder,}"]'|&grep newfstatat|wc -l
400
</code></pre>
<p>Because current glob implementation handles braces in early stage,<br>
then calls stat for each combination of brace patterns instead of using readdir results.</p>
<p>Below is a patch to use readdir instead of hundreds of syscalls.<br>
Note that this patch changes the order of the result as follows:</p>
<pre><code> 1) Failure:
TestDir#test_glob [/home/naruse/ruby/test/ruby/test_dir.rb:157]:
<["/tmp/test-all/__test_dir__20170906-8620-ka8yqu/}}{}",
"/tmp/test-all/__test_dir__20170906-8620-ka8yqu/}}a"]> expected but was
<["/tmp/test-all/__test_dir__20170906-8620-ka8yqu/}}a",
"/tmp/test-all/__test_dir__20170906-8620-ka8yqu/}}{}"]>.
</code></pre>
<pre><code>diff --git a/dir.c b/dir.c
index b7afaec4e0..7ca84fa8c5 100644
--- a/dir.c
+++ b/dir.c
@@ -290,6 +290,8 @@ bracket(
#define UNESCAPE(p) (escape && *(p) == '\\' ? (p) + 1 : (p))
#define ISEND(p) (!*(p) || (pathname && *(p) == '/'))
#define RETURN(val) return *pcur = p, *scur = s, (val);
+#define FNMATCH_ALLOC_N(type, n) ((type *)malloc(sizeof(type) * (n)))
+#define FNMATCH_FREE(ptr) free(ptr)
static int
fnmatch_helper(
@@ -348,6 +350,56 @@ fnmatch_helper(
}
goto failed;
}
+
+ case '{': {
+ size_t len = pend - p;
+ char *buf = FNMATCH_ALLOC_N(char, len);
+ const char *rbrace = NULL;
+ while (p < pend) {
+ const char *t = ++p;
+ int nest = 0;
+ while (p < pend && !(*p == ',' && nest == 0)) {
+ if (*p == '{') nest++;
+ if (*p == '}') {
+ if (nest == 0) {
+ if (!rbrace) rbrace = p;
+ goto rest;
+ }
+ nest--;
+ }
+ if (*p == '\\' && escape) {
+ if (++p >= pend) break;
+ }
+ Inc(p, pend, enc);
+ }
+ if (!rbrace) {
+ rbrace = p;
+ while (rbrace < pend && !(*rbrace == '}' && nest == 0)) {
+ if (*rbrace == '{') nest++;
+ if (*rbrace == '}') nest--;
+ if (*rbrace == '\\' && escape) {
+ if (++p >= pend) break;
+ }
+ Inc(rbrace, pend, enc);
+ }
+ }
+rest:
+ memcpy(buf, t, p-t);
+ buf[p-t]=0;
+ strlcpy(buf+(p-t), rbrace+1, len-(p-t));
+ {
+ const char *pp = buf, *ss = s;
+ r = fnmatch_helper((const char **)&pp, &ss, flags|FNM_DOTMATCH, enc);
+ }
+ if (r == 0) {
+ p = buf;
+ FNMATCH_FREE(buf);
+ RETURN(0);
+ }
+ if (p >= rbrace) break;
+ }
+ FNMATCH_FREE(buf);
+ }
}
/* ordinary */
@@ -1426,6 +1478,12 @@ has_magic(const char *p, const char *pend, int flags, rb_encoding *enc)
case '[':
return MAGICAL;
+ case '{':
+ if (flags & FNM_EXTGLOB) {
+ return MAGICAL;
+ }
+ break;
+
case '\\':
if (escape && p++ >= pend)
continue;
@@ -2272,6 +2330,13 @@ push_pattern(const char *path, VALUE ary, void *enc)
rb_ary_push(ary, name);
}
+struct push_glob_args {
+ struct glob_args glob;
+ int flags;
+ int fd;
+};
+static int push_caller(const char *path, VALUE val, void *enc);
+
static int
ruby_brace_expand(const char *str, int flags, ruby_glob_func *func, VALUE arg,
rb_encoding *enc, VALUE var)
@@ -2280,7 +2345,7 @@ ruby_brace_expand(const char *str, int flags, ruby_glob_func *func, VALUE arg,
const char *p = str;
const char *pend = p + strlen(p);
const char *s = p;
- const char *lbrace = 0, *rbrace = 0;
+ const char *lbrace = NULL, *rbrace = NULL;
int nest = 0, status = 0;
while (*p) {
@@ -2299,9 +2364,18 @@ ruby_brace_expand(const char *str, int flags, ruby_glob_func *func, VALUE arg,
if (lbrace && rbrace) {
size_t len = strlen(s) + 1;
- char *buf = GLOB_ALLOC_N(char, len);
+ char *buf;
long shift;
+ if (func == push_caller && !strchr(lbrace, '/')) {
+ /* Now it reaches file basename entry. */
+ /* Handle braces in glob_helper */
+ struct push_glob_args *a = (struct push_glob_args *)arg;
+ a->flags |= FNM_EXTGLOB;
+ return glob_call_func(func, s, arg, enc);
+ }
+
+ buf = GLOB_ALLOC_N(char, len);
if (!buf) return -1;
memcpy(buf, s, lbrace-s);
shift = (lbrace-s);
@@ -2365,12 +2439,6 @@ ruby_brace_glob(const char *str, int flags, ruby_glob_func *func, VALUE arg)
return ruby_brace_glob_with_enc(str, flags, func, arg, rb_ascii8bit_encoding());
}
-struct push_glob_args {
- struct glob_args glob;
- int flags;
- int fd;
-};
-
static int
push_caller(const char *path, VALUE val, void *enc)
{
</code></pre>
Ruby master - Feature #13869 (Open): Filter non directories from Dir.glob
https://redmine.ruby-lang.org/issues/13869
2017-09-05T14:30:58Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Dir.glob is a tool to fetch filesystem entries with filtering.</p>
<p>On Rails, it often query files from template directories with braces (FNM_EXTGLOB)<br>
<a href="https://github.com/rails/rails/blob/6f1c18308ebffc97d51440cdeed7be71de58f26a/actionview/lib/action_view/template/resolver.rb#L247" class="external">https://github.com/rails/rails/blob/6f1c18308ebffc97d51440cdeed7be71de58f26a/actionview/lib/action_view/template/resolver.rb#L247</a></p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="k">def</span> <span class="nf">find_template_paths</span><span class="p">(</span><span class="n">query</span><span class="p">)</span>
<span class="no">Dir</span><span class="p">[</span><span class="n">query</span><span class="p">].</span><span class="nf">uniq</span><span class="p">.</span><span class="nf">reject</span> <span class="k">do</span> <span class="o">|</span><span class="n">filename</span><span class="o">|</span>
<span class="no">File</span><span class="p">.</span><span class="nf">directory?</span><span class="p">(</span><span class="n">filename</span><span class="p">)</span> <span class="o">||</span>
<span class="c1"># deals with case-insensitive file systems.</span>
<span class="o">!</span><span class="no">File</span><span class="p">.</span><span class="nf">fnmatch</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="n">filename</span><span class="p">,</span> <span class="no">File</span><span class="o">::</span><span class="no">FNM_EXTGLOB</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
<p>By this code, File.directory?() will call lstat(2) system call for each files.</p>
<p>But if Dir.glob is extended, it can avoid calling lstat because it can fetch entry's type from struct dirent<br>
(on many platforms).</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/dir.c b/dir.c
index b7afaec4e0..5db9c2dc79 100644
</span><span class="gd">--- a/dir.c
</span><span class="gi">+++ b/dir.c
</span><span class="p">@@ -206,6 +206,7 @@</span> typedef enum {
#else
#define FNM_SHORTNAME 0
#endif
<span class="gi">+#define FNM_NONDIR 0x40
</span>
#define FNM_NOMATCH 1
#define FNM_ERROR 2
<span class="p">@@ -1408,7 +1409,7 @@</span> do_opendir(const int basefd, const char *path, int flags, rb_encoding *enc,
}
/* Globing pattern */
<span class="gd">-enum glob_pattern_type { PLAIN, ALPHA, MAGICAL, RECURSIVE, MATCH_ALL, MATCH_DIR };
</span><span class="gi">+enum glob_pattern_type { PLAIN, ALPHA, MAGICAL, RECURSIVE, MATCH_ALL, MATCH_DIR, MATCH_NONDIR };
</span>
/* Return nonzero if S has any special globbing chars in it. */
static enum glob_pattern_type
<span class="p">@@ -1581,7 +1582,7 @@</span> glob_make_pattern(const char *p, const char *e, int flags, rb_encoding *enc)
glob_free_pattern(list);
return 0;
}
<span class="gd">- tmp->type = dirsep ? MATCH_DIR : MATCH_ALL;
</span><span class="gi">+ tmp->type = dirsep ? MATCH_DIR : flags & flags & FNM_NONDIR ? MATCH_NONDIR : MATCH_ALL;
</span> tmp->str = 0;
*tail = tmp;
tmp->next = 0;
<span class="p">@@ -1893,7 +1894,7 @@</span> glob_helper(
struct stat st;
int status = 0;
struct glob_pattern **cur, **new_beg, **new_end;
<span class="gd">- int plain = 0, magical = 0, recursive = 0, match_all = 0, match_dir = 0;
</span><span class="gi">+ int plain = 0, magical = 0, recursive = 0, match_all = 0, match_dir = 0, match_nondir = 0;
</span> int escape = !(flags & FNM_NOESCAPE);
size_t pathlen = baselen + namelen;
const char *base = path;
<span class="p">@@ -1926,6 +1927,9 @@</span> glob_helper(
case MATCH_DIR:
match_dir = 1;
break;
<span class="gi">+ case MATCH_NONDIR:
+ match_nondir = 1;
+ break;
</span> case RECURSIVE:
rb_bug("continuous RECURSIVEs");
}
<span class="p">@@ -1940,7 +1944,7 @@</span> glob_helper(
pathtype = path_noent;
}
}
<span class="gd">- if (match_dir && (pathtype == path_unknown || pathtype == path_symlink)) {
</span><span class="gi">+ if ((match_dir || match_nondir) && (pathtype == path_unknown || pathtype == path_symlink)) {
</span> if (do_stat(fd, base, &st, flags, enc) == 0) {
pathtype = IFTODT(st.st_mode);
}
<span class="p">@@ -1953,6 +1957,11 @@</span> glob_helper(
status = glob_call_func(funcs->match, subpath, arg, enc);
if (status) return status;
}
<span class="gi">+ if (match_nondir && pathtype > path_noent && pathtype != path_directory) {
+ const char *subpath = path + baselen + (baselen && path[baselen] == '/');
+ status = glob_call_func(funcs->match, subpath, arg, enc);
+ if (status) return status;
+ }
</span> if (match_dir && pathtype == path_directory) {
const char *subpath = path + baselen + (baselen && path[baselen] == '/');
char *tmp = join_path(subpath, namelen, dirsep, "", 0);
<span class="p">@@ -3152,4 +3161,10 @@</span> Init_Dir(void)
* on Microsoft Windows.
*/
rb_file_const("FNM_SHORTNAME", INT2FIX(FNM_SHORTNAME));
<span class="gi">+
+ /* Document-const: File::Constants::FNM_NONDIR
+ *
+ * Makes patterns to match non directory. Valid only
+ */
+ rb_file_const("FNM_NONDIR", INT2FIX(FNM_NONDIR));
</span> }
</code></pre>
Ruby master - Bug #13852 (Closed): Backport r59693,59695 (Net::HTTP should raise error when CR/LF...
https://redmine.ruby-lang.org/issues/13852
2017-08-30T17:25:44Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>This is a ticket for backport management. The issue was already fixed on trunk.<br>
Let's treat r59693 as a bugfix and backport into stable branches.</p>
Ruby master - Feature #13712 (Closed): String#start_with? with regexp
https://redmine.ruby-lang.org/issues/13712
2017-07-04T09:18:51Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>String#start_with? should receive regexp.</p>
<p>When I write a parser, I want to check a string is start with a pattern or not.<br>
It's just the same thing with <a href="https://ruby-doc.org/stdlib-2.4.0/libdoc/strscan/rdoc/StringScanner.html#method-i-match-3F" class="external">StringScanner#match</a></p>
<p>If I want to do the same thing with normal string method, it needs to write like <code>/\A#{re}/.match(…)</code>.<br>
But if re is argument, it needs to create a new temporary regexp every time.</p>
<p>Though we have a workaround as follows but it's bit tricky.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="s2">"foo "</span><span class="p">.</span><span class="nf">rindex</span><span class="p">(</span><span class="sr">/fo+./</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
</code></pre>
<p>A patch is following:</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/re.c b/re.c
index d0aa2a792e..f672ba75ec 100644
</span><span class="gd">--- a/re.c
</span><span class="gi">+++ b/re.c
</span><span class="p">@@ -1588,6 +1588,84 @@</span> rb_reg_search(VALUE re, VALUE str, long pos, int reverse)
return rb_reg_search0(re, str, pos, reverse, 1);
}
<span class="gi">+bool
+rb_reg_start_with_p(VALUE re, VALUE str)
+{
+ long pos = 0;
+ long result;
+ VALUE match;
+ struct re_registers regi, *regs = &regi;
+ regex_t *reg;
+ int tmpreg;
+ onig_errmsg_buffer err = "";
+
+ reg = rb_reg_prepare_re0(re, str, err);
+ tmpreg = reg != RREGEXP_PTR(re);
+ if (!tmpreg) RREGEXP(re)->usecnt++;
+
+ match = rb_backref_get();
+ if (!NIL_P(match)) {
+ if (FL_TEST(match, MATCH_BUSY)) {
+ match = Qnil;
+ }
+ else {
+ regs = RMATCH_REGS(match);
+ }
+ }
+ if (NIL_P(match)) {
+ MEMZERO(regs, struct re_registers, 1);
+ }
+ result = onig_match(reg,
+ (UChar*)(RSTRING_PTR(str)),
+ ((UChar*)(RSTRING_PTR(str)) + RSTRING_LEN(str)),
+ (UChar*)(RSTRING_PTR(str)),
+ regs, ONIG_OPTION_NONE);
+ if (!tmpreg) RREGEXP(re)->usecnt--;
+ if (tmpreg) {
+ if (RREGEXP(re)->usecnt) {
+ onig_free(reg);
+ }
+ else {
+ onig_free(RREGEXP_PTR(re));
+ RREGEXP_PTR(re) = reg;
+ }
+ }
+ if (result < 0) {
+ if (regs == &regi)
+ onig_region_free(regs, 0);
+ if (result == ONIG_MISMATCH) {
+ rb_backref_set(Qnil);
+ return false;
+ }
+ else {
+ onig_error_code_to_str((UChar*)err, (int)result);
+ rb_reg_raise(RREGEXP_SRC_PTR(re), RREGEXP_SRC_LEN(re), err, re);
+ }
+ }
+
+ if (NIL_P(match)) {
+ int err;
+ match = match_alloc(rb_cMatch);
+ err = rb_reg_region_copy(RMATCH_REGS(match), regs);
+ onig_region_free(regs, 0);
+ if (err) rb_memerror();
+ }
+ else {
+ FL_UNSET(match, FL_TAINT);
+ }
+
+ RMATCH(match)->str = rb_str_new4(str);
+ OBJ_INFECT(match, str);
+
+ RMATCH(match)->regexp = re;
+ RMATCH(match)->rmatch->char_offset_updated = 0;
+ rb_backref_set(match);
+
+ OBJ_INFECT(match, re);
+
+ return true;
+}
+
</span> VALUE
rb_reg_nth_defined(int nth, VALUE match)
{
<span class="gh">diff --git a/string.c b/string.c
index 072f1329ee..6542a4acb1 100644
</span><span class="gd">--- a/string.c
</span><span class="gi">+++ b/string.c
</span><span class="p">@@ -9126,6 +9126,7 @@</span> rb_str_rpartition(VALUE str, VALUE sep)
RSTRING_LEN(str)-pos-RSTRING_LEN(sep)));
}
<span class="gi">+extern bool rb_reg_start_with_p(VALUE re, VALUE str);
</span> /*
* call-seq:
* str.start_with?([prefixes]+) -> true or false
<span class="p">@@ -9146,11 +9147,20 @@</span> rb_str_start_with(int argc, VALUE *argv, VALUE str)
for (i=0; i<argc; i++) {
VALUE tmp = argv[i];
<span class="gd">- StringValue(tmp);
- rb_enc_check(str, tmp);
- if (RSTRING_LEN(str) < RSTRING_LEN(tmp)) continue;
- if (memcmp(RSTRING_PTR(str), RSTRING_PTR(tmp), RSTRING_LEN(tmp)) == 0)
- return Qtrue;
</span><span class="gi">+ switch (BUILTIN_TYPE(tmp)) {
+ case T_REGEXP:
+ {
+ bool r = rb_reg_start_with_p(tmp, str);
+ if (r) return Qtrue;
+ }
+ break;
+ default:
+ StringValue(tmp);
+ rb_enc_check(str, tmp);
+ if (RSTRING_LEN(str) < RSTRING_LEN(tmp)) continue;
+ if (memcmp(RSTRING_PTR(str), RSTRING_PTR(tmp), RSTRING_LEN(tmp)) == 0)
+ return Qtrue;
+ }
</span> }
return Qfalse;
}
</code></pre>
Ruby master - Feature #13577 (Assigned): Digest.file accidentally receives File object but uses f...
https://redmine.ruby-lang.org/issues/13577
2017-05-19T09:03:19Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Digest::SHA256.file()'s first argument is file path name but it accidentally accepts file object.<br>
But for file objects created with O_TMPFILE to_path returns the directory of the temporary file and this File.open will fail.</p>
<pre><code> class ::Digest::Class
# Creates a digest object and reads a given file, _name_.
# Optional arguments are passed to the constructor of the digest
# class.
#
# p Digest::SHA256.file("X11R6.8.2-src.tar.bz2").hexdigest
# # => "f02e3c85572dc9ad7cb77c2a638e3be24cc1b5bea9fdbb0b0299c9668475c534"
def self.file(name, *args)
new(*args).file(name)
end
end
module Instance
# Updates the digest with the contents of a given file _name_ and
# returns self.
def file(name)
File.open(name, "rb") {|f|
buf = ""
while f.read(16384, buf)
update buf
end
}
self
end
</code></pre>
Ruby master - Bug #13304 (Closed): public function rb_thread_fd_close is removed at r57422
https://redmine.ruby-lang.org/issues/13304
2017-03-12T19:06:16Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>It hits abi-check:<br>
<a href="http://rubyci.org/logs/www.rubyist.net/~akr/chkbuild/debian/ruby-2.4/log/20170312T170653Z.log.html.gz#abi-check" class="external">http://rubyci.org/logs/www.rubyist.net/~akr/chkbuild/debian/ruby-2.4/log/20170312T170653Z.log.html.gz#abi-check</a></p>
Ruby master - Bug #13150 (Closed): TestMarshal failures on FreeBSD with gcc7 because of GC
https://redmine.ruby-lang.org/issues/13150
2017-01-23T16:58:04Z
naruse (Yui NARUSE)
naruse@airemix.jp
<pre><code> 1) Failure:
TestMarshal#test_context_switch [/home/naruse/ruby/test/ruby/test_marshal.rb:368]:
[StopIteration] exception expected, not.
Class: <RuntimeError>
Message: <"Marshal.dump reentered at marshal_dump">
---Backtrace---
/home/naruse/ruby/test/ruby/test_marshal.rb:345:in `dump'
/home/naruse/ruby/test/ruby/test_marshal.rb:345:in `dump_each'
../../ruby/test/runner.rb: TestMarshal#test_context_switch:1:in `each'
---------------
2) Failure:
TestMarshal#test_gc [/home/naruse/ruby/test/ruby/test_marshal.rb:187]:
Exception raised:
<#<RuntimeError: Marshal.dump reentered at _dump>>.
Finished tests in 0.153251s, 698.2016 tests/s, 6238.1380 assertions/s.
107 tests, 956 assertions, 2 failures, 0 errors, 0 skips
ruby -v: ruby 2.5.0dev (2017-01-23 trunk 57407) [x86_64-freebsd10.3]
</code></pre>
Ruby master - Misc #13027 (Closed): Release Engineering 2.4
https://redmine.ruby-lang.org/issues/13027
2016-12-12T14:43:37Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>META ticket for Release Engineering 2.4.</p>
<p>see also <a class="version" href="https://redmine.ruby-lang.org/versions/33">2.4</a></p>
<ul>
<li>current status: feature freeze</li>
<li>Q: Can I add a new feature?
<ul>
<li>A: No. Its status is already "feature freeze".</li>
</ul>
</li>
<li>Q: Can I fix a bug?
<ul>
<li>A: Yes</li>
</ul>
</li>
<li>Q: I want to complain about 2.4.0-rc1
<ul>
<li>A: create a ticket. It may change 2.4.0 RTM.</li>
</ul>
</li>
<li>Q: When ruby_2_4 branch is created?
<ul>
<li>A: scheduled on 2016-12-23</li>
</ul>
</li>
<li>Q: When Ruby 2.4.0 is released?
<ul>
<li>A: scheduled at 2016-12-25T00:00:00/03:00:00</li>
</ul>
</li>
</ul>
Ruby master - Feature #13020 (Closed): Zlib.gzip and Zlib.gunzip
https://redmine.ruby-lang.org/issues/13020
2016-12-09T22:37:22Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>I added Zlib.deflate/inflate [Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Add Zlib.deflate / Zlib.inflate (Closed)" href="https://redmine.ruby-lang.org/issues/4180">#4180</a>] before, but writing/reading gzip is still too complex.<br>
It should have shorthand method.</p>
Ruby master - Bug #12923 (Closed): Accessing singleton_class of fstring cause assertion failure
https://redmine.ruby-lang.org/issues/12923
2016-11-11T17:57:23Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>以下のワンライナーが「Assertion Failed: string.c:343:register_fstring:RBASIC_CLASS(ret) == rb_cString」します。</p>
<p>./miniruby -e'#encoding:us-ascii' -e'ObjectSpace.each_object{|o| o.singleton_class if o.is_a?(String)}; "hoge".intern'</p>
<p>クラッシュログは自分で走らせてもらうとして、何が起きているかというと、<br>
(1) 文字列リテラルをはじめとして、何らかのかたちでシンボルを作らずにrb_fstringからfrozenなStringを作る<br>
(2) その文字列に対してsingleton_classを呼ぶ。するとそのRVALUE->klassにsingleton_classが代入される<br>
(3) 同じ内容の文字列でString#internする(1.でシンボルを作っているとlookup_str_sym()にひっかかる)<br>
(4) register_fstringの"assert(RBASIC_CLASS(ret) == rb_cString);"で落ちる</p>
<p>しかし、どう直しましょうかね。</p>
Ruby master - Feature #12896 (Closed): Add compiler version message into rbconfig
https://redmine.ruby-lang.org/issues/12896
2016-11-04T06:30:53Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>When a extension library developer receives bug reports, they sometimes want to know what compiler is used for building the ruby binary.<br>
We can already know its compiler option but cannot know compiler name and version.</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/configure.in b/configure.in
index 3734afa..6793d7d 100644
</span><span class="gd">--- a/configure.in
</span><span class="gi">+++ b/configure.in
</span><span class="p">@@ -536,8 +536,10 @@</span> for option in --version -v -V -qversion; do
AS_CASE($cc_version_status, [0], [:], [continue])
AS_CASE($cc_version_message, [*Warning*], [continue])
cc_version='$(CC) '$option
<span class="gi">+ break
</span> done
AC_SUBST(CC_VERSION, $cc_version)
<span class="gi">+AC_SUBST(CC_VERSION_MESSAGE, $cc_version_message)
</span><span class="err">
</span> RUBY_UNIVERSAL_ARCH
if test "$target_cpu" != "$host_cpu" -a "$GCC" = yes -a "$cross_compiling" = no -a "$universal_binary" = no; then
</code></pre>
Ruby master - Bug #12791 (Closed): Don't allow ,-separator for cookie
https://redmine.ruby-lang.org/issues/12791
2016-09-27T03:11:40Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>RFC2965 allowed both ; and , as a separator for cookie, but RFC6265 only allows ;.</p>
<p>Moreover CVE-2016-7401 uses , as a separator to overwrite CSRF-token.<br>
<a href="https://gist.github.com/mala/457a25650950d4daf4144f98159802cc" class="external">https://gist.github.com/mala/457a25650950d4daf4144f98159802cc</a></p>
Ruby master - Misc #12751 (Open): Incompatibility of Ruby 3
https://redmine.ruby-lang.org/issues/12751
2016-09-12T06:38:38Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>META ticket for Ruby 3's breakages</p>
<ul>
<li>Encoding on Windows
<ul>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: On Windows use UTF-8 as filesystem encoding (Closed)" href="https://redmine.ruby-lang.org/issues/12654">#12654</a>]</li>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Use UTF-8 encoding for ENV on Windows (Closed)" href="https://redmine.ruby-lang.org/issues/12650">#12650</a>]</li>
</ul>
</li>
</ul>
Ruby master - Bug #12711 (Closed): Darwin doesn't show C backtrace correctly if iSIGSEGV is recei...
https://redmine.ruby-lang.org/issues/12711
2016-08-29T18:38:08Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Current Ruby can show C backtrace on the following case</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="no">Process</span><span class="p">.</span><span class="nf">kill</span> <span class="ss">:SEGV</span><span class="p">,</span> <span class="vg">$$</span>
</code></pre>
<p>But can't on the following:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="nb">require</span><span class="s2">"fiddle"</span>
<span class="no">Fiddle</span><span class="p">.</span><span class="nf">dlunwrap</span><span class="p">(</span><span class="mi">100</span><span class="p">).</span><span class="nf">class</span>
</code></pre>
Ruby master - Bug #12568 (Closed): wrong ArgumentError if an array is given for instance_exec wit...
https://redmine.ruby-lang.org/issues/12568
2016-07-07T18:50:32Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>From Ruby 2.2 to trunk, it wrongly raise ArgumentError as follows:</p>
<p>Sample code:</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">instance_exec</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">]){</span><span class="o">|</span><span class="n">a</span><span class="o">|</span> <span class="nb">p</span> <span class="n">a</span><span class="p">}</span>
<span class="n">instance_exec</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">]){</span><span class="o">|</span><span class="n">a</span><span class="o">=</span><span class="p">[]</span><span class="o">|</span> <span class="nb">p</span> <span class="n">a</span><span class="p">}</span>
<span class="n">instance_exec</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">],</span> <span class="o">&-></span><span class="p">(</span><span class="n">a</span><span class="p">){</span> <span class="nb">p</span> <span class="n">a</span> <span class="p">})</span>
<span class="o">-></span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="p">[]){</span> <span class="nb">p</span> <span class="n">a</span> <span class="p">}.</span><span class="nf">to_proc</span><span class="p">.</span><span class="nf">call</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">])</span>
<span class="n">instance_exec</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">],</span> <span class="o">&-></span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="p">[]){</span> <span class="nb">p</span> <span class="n">a</span> <span class="p">})</span>
</code></pre>
<p>Expected result:</p>
<pre><code>[1, 2, 3]
[1, 2, 3]
[1, 2, 3]
[1, 2, 3]
[1, 2, 3]
</code></pre>
<p>Actual result:</p>
<pre><code>[1, 2, 3]
[1, 2, 3]
[1, 2, 3]
[1, 2, 3]
test.rb:7:in `block in <main>': wrong number of arguments (given 3, expected 0..1) (ArgumentError)
from test.rb:7:in `instance_exec'
from test.rb:7:in `<main>'
</code></pre>
<p>This issue affect <a href="https://github.com/rails/rails/pull/25699" class="external">https://github.com/rails/rails/pull/25699</a></p>
Ruby master - Bug #12563 (Closed): backport 49758,50356
https://redmine.ruby-lang.org/issues/12563
2016-07-06T20:40:07Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>49758,50356 is required to run <code>make test-rubyspec</code> because without them rubyspec fails to build option/capi/ext.<br>
<a href="https://travis-ci.org/ruby/ruby/builds/142788121" class="external">https://travis-ci.org/ruby/ruby/builds/142788121</a></p>
Ruby master - Bug #12560 (Closed): backport r55602
https://redmine.ruby-lang.org/issues/12560
2016-07-06T18:20:29Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>They may fail parallel test-all.</p>
<ul>
<li>ruby_2_3: <a href="https://travis-ci.org/ruby/ruby/builds/141708115" class="external">https://travis-ci.org/ruby/ruby/builds/141708115</a>
</li>
<li>ruby_2_2: <a href="https://travis-ci.org/ruby/ruby/builds/142772798" class="external">https://travis-ci.org/ruby/ruby/builds/142772798</a>
</li>
</ul>
Ruby master - Feature #12553 (Closed): IO.readlines(filename, chomp: true)
https://redmine.ruby-lang.org/issues/12553
2016-07-05T09:11:11Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>IO.readlinesやIO.foreach、IO#each_lineなどは戻り値(またはブロックパラメータ)のそれぞれの「行」に改行を含みます。</p>
<p>POSIXの "Line" の定義は末尾の改行を含んでのものなので非常に正しい挙動なのですが、正直不便です。</p>
<blockquote>
<p><a href="http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_206" class="external">3.206 Line</a></p>
<p>A sequence of zero or more non- characters plus a terminating character.</p>
</blockquote>
<p>例えば IO.readlines(filename, chomp: true) などで改行を最初から削ってくれませんか</p>
Ruby master - Feature #12336 (Closed): Use -std=gnu99 instead of -std=c99 for GCC
https://redmine.ruby-lang.org/issues/12336
2016-05-01T09:58:53Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>r36038以降、Ruby本体をビルドする際にはGCCに-stdオプションを与えており、<br>
若干の変更を経て現在では-std=iso9899:1999 (-std=c99相当)を指定しています。</p>
<p>しかしこの指定の場合、GCCのC標準関数の代わりにビルトイン関数を用いる最適化が行われないため、<br>
現在ちょっと残念な状態が続いています。<br>
<a href="https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html" class="external">https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html</a></p>
<p>また、これの導入のきっかけはおそらくr36029の</p>
<ul>
<li>compile.c (iseq_set_sequence): nonstatic initializer of an aggregate type is a C99ism.</li>
<li>compile.c (enum compile_array_type_t): comma at the end of enum list is a C99ism.</li>
</ul>
<p>だと思うのですが、すでにc99相当なのであんまり効いてない……かな?<br>
もっともやりがちなMixed Declarationsは別に-Wdeclaration-after-statementがあるのでそれはそれで問題ありません。<br>
<a href="https://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html" class="external">https://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html</a><br>
<a href="https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html" class="external">https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html</a></p>
<p>現在だと、-std=gnu99でおおむね同レベルの警告を維持できると同時に、<br>
GCCのビルトイン関数を使えるようになってわずかな手間でちょっと早くなって嬉しいと思うのですがいかがでしょう。</p>
Ruby master - Bug #12320 (Closed): Skip SHA from test_digest_constants for LibreSSL 2.3
https://redmine.ruby-lang.org/issues/12320
2016-04-26T06:30:04Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Backport r53718 to fix the first one of ruby/openssl#40<br>
<a href="https://github.com/ruby/openssl/issues/40#issuecomment-159839338" class="external">https://github.com/ruby/openssl/issues/40#issuecomment-159839338</a></p>
Ruby master - Misc #12283 (Closed): Obsolete ChangeLog and commit message in Git-style
https://redmine.ruby-lang.org/issues/12283
2016-04-14T18:57:41Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>How about Git-style commit message instead of ChangeLog?</p>
<a name="Proposal"></a>
<h2 >Proposal<a href="#Proposal" class="wiki-anchor">¶</a></h2>
<p>After we reach the consensus</p>
<ol>
<li>Write a commit message in Git-style.</li>
</ol>
<ul>
<li>Line 1: summary of commit (around 80 characters or less, <50 is better)</li>
<li>Line 2: empty line</li>
<li>Line 3-: More detailed explanation of commit</li>
</ul>
<p>See also: <a href="https://git-scm.com/book/ch5-2.html#Commit-Guidelines" class="external">https://git-scm.com/book/ch5-2.html#Commit-Guidelines</a></p>
<ol start="2">
<li>Stop manually edit ChangeLog and auto-generate on <code>make dist</code>.</li>
</ol>
<a name="Rationale"></a>
<h2 >Rationale<a href="#Rationale" class="wiki-anchor">¶</a></h2>
<p>A good commit message should include the objective of the commit.<br>
Committers should write "what purpose the change is for".<br>
In this sense, ChangeLog is not a good format. It lists per-file changes, so committers tend to write just "how the files were changed".</p>
<p>Also, ChangeLog causes commit conflicts frequently.<br>
Though it is not related with the actual commit.<br>
People who uses svn/git should see the log with it; ChangeLog is only for tarball users.</p>
<a name="Status"></a>
<h2 >Status<a href="#Status" class="wiki-anchor">¶</a></h2>
<p>mame made this proposal at the developers' meeting (13 Apr.), and there was no objection.</p>
<p>Matz agreed with this proposal, provided that a ChangeLog-like plain-text file is generated from the commit log.<br>
The release manager will create the file when Ruby is released.</p>
<a name="Note"></a>
<h3 >Note<a href="#Note" class="wiki-anchor">¶</a></h3>
<p>This ticket does NOT propose a migration from SVN to Git. Never discuss that in this ticket.</p>
Ruby master - Feature #12244 (Open): Add a way to `integer - integer % num`
https://redmine.ruby-lang.org/issues/12244
2016-04-02T13:45:33Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>We sometimes calculates <code>integer - integer % num</code>.</p>
<p>For example time series events into time partitions, we write code like</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">event</span> <span class="c1"># {time: 1459580435, name: "hoge", text: "Rawr!"}</span>
<span class="n">partition</span> <span class="o">=</span> <span class="n">event</span><span class="p">[</span><span class="ss">:time</span><span class="p">]</span> <span class="o">-</span> <span class="n">event</span><span class="p">[</span><span class="ss">:time</span><span class="p">]</span> <span class="o">%</span> <span class="n">num</span>
<span class="n">chunk</span> <span class="o">=</span> <span class="n">get_chunk</span><span class="p">(</span><span class="n">partition</span><span class="p">)</span>
<span class="n">chunk</span><span class="p">.</span><span class="nf">write</span> <span class="n">event</span>
</code></pre>
<p><a href="https://twitter.com/tagomoris/status/715814050534461440" class="external">https://twitter.com/tagomoris/status/715814050534461440</a><br>
<a href="https://twitter.com/tagomoris/status/715814961260457985" class="external">https://twitter.com/tagomoris/status/715814961260457985</a></p>
<p>The name is always issue.<br>
There are some suggestions likes Integer#adjust](<a href="https://twitter.com/cocoatomo/status/716088708655489024" class="external">https://twitter.com/cocoatomo/status/716088708655489024</a>).</p>
<p>kosaki says <a href="https://twitter.com/kosaki55tea/status/716059296186765312" class="external">Excel's FLOOR() function is FLOOR(number, significance)</a>.<br>
Therefore Ruby should be <a href="https://twitter.com/kosaki55tea/status/716106530114768897" class="external">Integer#floor(digits=1, significance: nil)</a>.<br>
<a href="https://twitter.com/kosaki55tea/status/716107516409581568" class="external">kosaki agrees this</a>.</p>
<p>I checked the speed of a half baked implementation..., but it's 10x slow...</p>
<pre><code>% time ./miniruby -e'i=10000000;while i>0;i-=1;1459497599.floor(significance: 3600);end'
./miniruby 6.58s user 0.02s system 99% cpu 6.596 total
#3 1459604131 22:35:31 naruse@windy:~/obj/ruby
% time ./miniruby -e'i=10000000;while i>0;i-=1;t=1459497599;t-t%3600;end'
./miniruby -e'i=10000000;while i>0;i-=1;t=1459497599;t-t%3600;end' 0.52s user 0.00s system 99% cpu 0.520 total
</code></pre>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/numeric.c b/numeric.c
index 37217a1..f6acfe3 100644
</span><span class="gd">--- a/numeric.c
</span><span class="gi">+++ b/numeric.c
</span><span class="p">@@ -4123,6 +4123,42 @@</span> int_dotimes(VALUE num)
<span class="err">
</span> /*
* call-seq:
<span class="gi">+ * int.floor([ndigits]) -> integer or float
+ *
+ * Rounds +int+ to a given precision in decimal digits (default 0 digits).
+ *
+ * Precision may be negative. Returns a floating point number when +ndigits+
+ * is positive, +self+ for zero, and round down for negative.
+ *
+ * 1.round #=> 1
+ * 1.round(2) #=> 1.0
+ * 15.round(-1) #=> 20
+ */
+
+static VALUE
+int_floor(int argc, VALUE* argv, VALUE num)
+{
+ static ID keyword_ids[1];
+ VALUE kwargs[1], ndigits, opt;
+ if (!keyword_ids[0]) {
+ CONST_ID(keyword_ids[0], "significance");
+ }
+
+ rb_scan_args(argc, argv, "01:", &ndigits, &opt);
+ if (!NIL_P(opt)) {
+ VALUE factor;
+ long a, b;
+ rb_get_kwargs(opt, keyword_ids, 0, 1, kwargs);
+ factor = kwargs[0];
+ a = FIX2LONG(num);
+ b = FIX2LONG(factor);
+ return LONG2FIX(a - a % b);
+ }
+ return Qnil;
+}
+
+/*
+ * call-seq:
</span> * int.round([ndigits]) -> integer or float
*
* Rounds +int+ to a given precision in decimal digits (default 0 digits).
<span class="p">@@ -4321,7 +4357,7 @@</span> Init_Numeric(void)
rb_define_method(rb_cInteger, "to_i", int_to_i, 0);
rb_define_method(rb_cInteger, "to_int", int_to_i, 0);
rb_define_method(rb_cInteger, "to_f", int_to_f, 0);
<span class="gd">- rb_define_method(rb_cInteger, "floor", int_to_i, 0);
</span><span class="gi">+ rb_define_method(rb_cInteger, "floor", int_floor, -1);
</span> rb_define_method(rb_cInteger, "ceil", int_to_i, 0);
rb_define_method(rb_cInteger, "truncate", int_to_i, 0);
rb_define_method(rb_cInteger, "round", int_round, -1);
</code></pre>
Ruby master - Feature #12208 (Closed): Improve ri command
https://redmine.ruby-lang.org/issues/12208
2016-03-23T05:22:46Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Ruby has well documented in rdoc.<br>
It also has a frontend CLI.</p>
<p>But people seems not using <code>ri</code>...</p>
<p>At first, write your request here to improve ri command!</p>
Ruby master - Bug #12158 (Closed): Fixnum#% doesn't show its name on ZeroDivisionError
https://redmine.ruby-lang.org/issues/12158
2016-03-08T08:54:05Z
naruse (Yui NARUSE)
naruse@airemix.jp
<pre><code>% ruby -ve'p 12345 % 0'
ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-freebsd10.2]
-e:1:in `<main>': divided by 0 (ZeroDivisionError)
</code></pre>
<p>It should behave like</p>
<pre><code>% ruby -ve'p 12345 / 0'
ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-freebsd10.2]
-e:1:in `/': divided by 0 (ZeroDivisionError)
from -e:1:in `<main>'
</code></pre>
Ruby master - Bug #12011 (Closed): honor Marshal.load post proc value for TYPE_LINK
https://redmine.ruby-lang.org/issues/12011
2016-01-20T02:25:15Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Following test doesn't work.<br>
A patch also attached.<br>
Both of them are worked by nahi.</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/test/ruby/test_marshal.rb b/test/ruby/test_marshal.rb
index 482637f..262e7f6 100644
</span><span class="gd">--- a/test/ruby/test_marshal.rb
</span><span class="gi">+++ b/test/ruby/test_marshal.rb
</span><span class="p">@@ -712,4 +712,10 @@</span> def test_no_internal_ids
assert_predicate(status, :success?)
assert_equal(expected, out)
end
<span class="gi">+
+ def test_marshal_post_proc
+ str = 'x' # for link
+ obj = [str, str]
+ assert_equal(['X', 'X'], Marshal.load(Marshal.dump(obj), ->(v) { v == str ? v.upcase : v }))
+ end
</span> end
<span class="gh">diff --git a/marshal.c b/marshal.c
index d67ce87..d64e5ff 100644
</span><span class="gd">--- a/marshal.c
</span><span class="gi">+++ b/marshal.c
</span><span class="p">@@ -1569,7 +1569,7 @@</span> r_object0(struct load_arg *arg, int *ivp, VALUE extmod)
rb_raise(rb_eArgError, "dump format error (unlinked)");
}
v = (VALUE)link;
<span class="gd">- r_post_proc(v, arg);
</span><span class="gi">+ v = r_post_proc(v, arg);
</span> break;
case TYPE_IVAR:
</code></pre>
<p><a href="https://github.com/ruby/ruby/pull/1204" class="external">https://github.com/ruby/ruby/pull/1204</a></p>
Ruby master - Feature #12010 (Closed): Exclude dot and dotdot from Dir#each
https://redmine.ruby-lang.org/issues/12010
2016-01-20T02:17:16Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p><code>Dir#each</code> and <code>Dir#read</code> (including <code>Dir.entries</code>, <code>Dir.foreach</code> and other methods) return <code>"."</code> and <code>".."</code> at first.<br>
But through the all real use case <code>"."</code> and <code>".."</code> are useless.<br>
How about excluding them?</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/dir.c b/dir.c
index 193b5be..4c23a2d 100644
</span><span class="gd">--- a/dir.c
</span><span class="gi">+++ b/dir.c
</span><span class="p">@@ -699,6 +699,8 @@</span> fundamental_encoding_p(rb_encoding *enc)
#else
# define READDIR(dir, enc) readdir((dir))
#endif
<span class="gi">+#define DIR_IS_DOT_OR_DOTDOT(dp) ((dp)->d_name[0] == '.' && \
+ ((dp)->d_name[1] == '\0' || ((dp)->d_name[1] == '.' && (dp)->d_name[2] == '\0')))
</span>
/*
* call-seq:
<span class="p">@@ -720,13 +722,12 @@</span> dir_read(VALUE dir)
GetDIR(dir, dirp);
errno = 0;
<span class="gd">- if ((dp = READDIR(dirp->dir, dirp->enc)) != NULL) {
- return rb_external_str_new_with_enc(dp->d_name, NAMLEN(dp), dirp->enc);
- }
- else {
- if (errno != 0) rb_sys_fail(0);
- return Qnil; /* end of stream */
</span><span class="gi">+ while ((dp = READDIR(dirp->dir, dirp->enc)) != NULL) {
+ if (!DIR_IS_DOT_OR_DOTDOT(dp))
+ return rb_external_str_new_with_enc(dp->d_name, NAMLEN(dp), dirp->enc);
</span> }
<span class="gi">+ if (errno != 0) rb_sys_fail(0);
+ return Qnil; /* end of stream */
</span> }
/*
<span class="p">@@ -764,6 +765,7 @@</span> dir_each(VALUE dir)
const char *name = dp->d_name;
size_t namlen = NAMLEN(dp);
VALUE path;
<span class="gi">+ if (DIR_IS_DOT_OR_DOTDOT(dp)) continue;
</span> #if NORMALIZE_UTF8PATH
if (norm_p && has_nonascii(name, namlen) &&
!NIL_P(path = rb_str_normalize_ospath(name, namlen))) {
<span class="gh">diff --git a/test/pathname/test_pathname.rb b/test/pathname/test_pathname.rb
index 2690a3f..33f0d44 100644
</span><span class="gd">--- a/test/pathname/test_pathname.rb
</span><span class="gi">+++ b/test/pathname/test_pathname.rb
</span><span class="p">@@ -1238,7 +1238,7 @@</span> def test_entries
with_tmpchdir('rubytest-pathname') {|dir|
open("a", "w") {}
open("b", "w") {}
<span class="gd">- assert_equal([Pathname("."), Pathname(".."), Pathname("a"), Pathname("b")], Pathname(".").entries.sort)
</span><span class="gi">+ assert_equal([Pathname("a"), Pathname("b")], Pathname(".").entries.sort)
</span> }
end
<span class="p">@@ -1248,7 +1248,7 @@</span> def test_each_entry
open("b", "w") {}
a = []
Pathname(".").each_entry {|v| a << v }
<span class="gd">- assert_equal([Pathname("."), Pathname(".."), Pathname("a"), Pathname("b")], a.sort)
</span><span class="gi">+ assert_equal([Pathname("a"), Pathname("b")], a.sort)
</span> }
end
<span class="p">@@ -1278,7 +1278,7 @@</span> def test_opendir
Pathname(".").opendir {|d|
d.each {|e| a << e }
}
<span class="gd">- assert_equal([".", "..", "a", "b"], a.sort)
</span><span class="gi">+ assert_equal(["a", "b"], a.sort)
</span> }
end
<span class="gh">diff --git a/test/ruby/test_dir.rb b/test/ruby/test_dir.rb
index 0cc5a6a..d3f6602 100644
</span><span class="gd">--- a/test/ruby/test_dir.rb
</span><span class="gi">+++ b/test/ruby/test_dir.rb
</span><span class="p">@@ -186,7 +186,7 @@</span> def test_glob_recursive
def assert_entries(entries)
entries.sort!
<span class="gd">- assert_equal(%w(. ..) + ("a".."z").to_a, entries)
</span><span class="gi">+ assert_equal(("a".."z").to_a, entries)
</span> end
def test_entries
</code></pre>
Ruby master - Feature #12005 (Closed): Unify Fixnum and Bignum into Integer
https://redmine.ruby-lang.org/issues/12005
2016-01-19T02:38:01Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>CRuby has two <code>Integer</code> classes, <code>Fixnum</code> and <code>Bignum</code>.<br>
But it is implementation detail.<br>
They should be seen as a single class <code>Integer</code> like <code>Flonum</code>.</p>
<a name="Compatibility-note"></a>
<h2 >Compatibility note<a href="#Compatibility-note" class="wiki-anchor">¶</a></h2>
<ul>
<li>Q: How do I check whether Fixnum and Bignum are unified or not?</li>
<li>A: check RUBY_INTEGER_UNIFICATION macro</li>
</ul>
Ruby master - Feature #11987 (Open): daemons can't show the backtrace of rb_bug
https://redmine.ruby-lang.org/issues/11987
2016-01-13T10:38:57Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Ruby shows backtrace and other information into stderr on rb_bug.<br>
But daemon process redirects stderr into /dev/null.<br>
How do I get the log?</p>
<p>NOTE: if I can reproduce this, I can use strace to get it. But issues are not always reproducible.</p>
Ruby master - Feature #11801 (Closed): rb_inspect shouldn't raise error even if calling inspect r...
https://redmine.ruby-lang.org/issues/11801
2015-12-10T04:22:06Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Current rb_inspect raises Encoding::Compatibility error if obj.inspect returns a string<br>
whose encoding is not compatible with resulted string's encoding.<br>
But the behavior is not useful on inspecting an array or a hash containing such strings<br>
because it prevents to dump them.</p>
<p>Following patch changes to escape such strings like String#inspect.</p>
<pre><code>diff --git a/object.c b/object.c
index d339ff9..1f73cb4 100644
--- a/object.c
+++ b/object.c
@@ -465,6 +465,7 @@ rb_any_to_s(VALUE obj)
return str;
}
+VALUE rb_str_escape(VALUE str);
/*
* If the default external encoding is ASCII compatible, the encoding of
* the inspected result must be compatible with it.
@@ -478,11 +479,11 @@ rb_inspect(VALUE obj)
rb_encoding *ext = rb_default_external_encoding();
if (!rb_enc_asciicompat(ext)) {
if (!rb_enc_str_asciionly_p(str))
- rb_raise(rb_eEncCompatError, "inspected result must be ASCII only if default external encoding is ASCII incompatible");
+ return rb_str_escape(str);
return str;
}
if (rb_enc_get(str) != ext && !rb_enc_str_asciionly_p(str))
- rb_raise(rb_eEncCompatError, "inspected result must be ASCII only or use the default external encoding");
+ return rb_str_escape(str);
return str;
}
diff --git a/string.c b/string.c
index e6df91d..319c516 100644
--- a/string.c
+++ b/string.c
@@ -5265,6 +5265,70 @@ rb_str_buf_cat_escaped_char(VALUE result, unsigned int c, int unicode_p)
return l;
}
+VALUE
+rb_str_escape(VALUE str)
+{
+ int encidx = ENCODING_GET(str);
+ rb_encoding *enc = rb_enc_from_index(encidx);
+ const char *p = RSTRING_PTR(str);
+ const char *pend = RSTRING_END(str);
+ const char *prev = p;
+ char buf[CHAR_ESC_LEN + 1];
+ VALUE result = rb_str_buf_new(0);
+ int unicode_p = rb_enc_unicode_p(enc);
+ int asciicompat = rb_enc_asciicompat(enc);
+
+ while (p < pend) {
+ unsigned int c, cc;
+ int n = rb_enc_precise_mbclen(p, pend, enc);
+ if (!MBCLEN_CHARFOUND_P(n)) {
+ if (p > prev) str_buf_cat(result, prev, p - prev);
+ n = rb_enc_mbminlen(enc);
+ if (pend < p + n)
+ n = (int)(pend - p);
+ while (n--) {
+ snprintf(buf, CHAR_ESC_LEN, "\\x%02X", *p & 0377);
+ str_buf_cat(result, buf, strlen(buf));
+ prev = ++p;
+ }
+ continue;
+ }
+ n = MBCLEN_CHARFOUND_LEN(n);
+ c = rb_enc_mbc_to_codepoint(p, pend, enc);
+ p += n;
+ switch (c) {
+ case '\n': cc = 'n'; break;
+ case '\r': cc = 'r'; break;
+ case '\t': cc = 't'; break;
+ case '\f': cc = 'f'; break;
+ case '\013': cc = 'v'; break;
+ case '\010': cc = 'b'; break;
+ case '\007': cc = 'a'; break;
+ case 033: cc = 'e'; break;
+ default: cc = 0; break;
+ }
+ if (cc) {
+ if (p - n > prev) str_buf_cat(result, prev, p - n - prev);
+ buf[0] = '\\';
+ buf[1] = (char)cc;
+ str_buf_cat(result, buf, 2);
+ prev = p;
+ }
+ else if (asciicompat && rb_enc_isascii(c, enc) && ISPRINT(c)) {
+ }
+ else {
+ if (p - n > prev) str_buf_cat(result, prev, p - n - prev);
+ rb_str_buf_cat_escaped_char(result, c, unicode_p);
+ prev = p;
+ }
+ }
+ if (p > prev) str_buf_cat(result, prev, p - prev);
+ ENCODING_CODERANGE_SET(result, rb_usascii_encindex(), ENC_CODERANGE_7BIT);
+
+ OBJ_INFECT_RAW(result, str);
+ return result;
+}
+
/*
* call-seq:
* str.inspect -> string
diff --git a/test/ruby/test_m17n.rb b/test/ruby/test_m17n.rb
index ca25f85..2e9e65b 100644
--- a/test/ruby/test_m17n.rb
+++ b/test/ruby/test_m17n.rb
@@ -278,7 +278,7 @@ def test_object_utf16_32_inspect
o = Object.new
[Encoding::UTF_16BE, Encoding::UTF_16LE, Encoding::UTF_32BE, Encoding::UTF_32LE].each do |e|
o.instance_eval "undef inspect;def inspect;'abc'.encode('#{e}');end"
- assert_raise(Encoding::CompatibilityError) { [o].inspect }
+ assert_equal '[abc]', [o].inspect
end
ensure
Encoding.default_internal = orig_int
@@ -302,13 +302,18 @@ def o.inspect
def o.inspect
"abc".encode(Encoding.default_external)
end
- assert_raise(Encoding::CompatibilityError) { [o].inspect }
+ assert_equal '[abc]', [o].inspect
Encoding.default_external = Encoding::US_ASCII
def o.inspect
"\u3042"
end
- assert_raise(Encoding::CompatibilityError) { [o].inspect }
+ assert_equal '[\u3042]', [o].inspect
+
+ def o.inspect
+ "\x82\xa0".force_encoding(Encoding::Windows_31J)
+ end
+ assert_equal '[\x{82A0}]', [o].inspect
ensure
Encoding.default_internal = orig_int
Encoding.default_external = orig_ext
diff --git a/core/array/shared/inspect.rb b/core/array/shared/inspect.rb
index dc30518..a169fd0 100644
--- a/core/array/shared/inspect.rb
+++ b/core/array/shared/inspect.rb
@@ -83,9 +83,9 @@ describe :array_inspect, shared: true do
it "raises if inspected result is not default external encoding" do
utf_16be = mock("utf_16be")
- utf_16be.should_receive(:inspect).and_return("utf_16be".encode!(Encoding::UTF_16BE))
+ utf_16be.should_receive(:inspect).and_return(%<"utf_16be \u3042">.encode!(Encoding::UTF_16BE))
- lambda { [utf_16be].send(@method) }.should raise_error(Encoding::CompatibilityError)
+ [utf_16be].send(@method).should == '["utf_16be \u3042"]'
end
end
end
</code></pre>
Ruby master - Bug #11669 (Closed): inconsitent behavior of refining frozen class
https://redmine.ruby-lang.org/issues/11669
2015-11-09T05:38:34Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Is this expected behavior?</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="k">class</span> <span class="nc">C</span>
<span class="k">def</span> <span class="nf">foo</span>
<span class="nb">p</span> <span class="mi">1</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">module</span> <span class="nn">Foo</span>
<span class="n">refine</span> <span class="no">C</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">foo</span>
<span class="nb">p</span> <span class="mi">2</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">using</span> <span class="no">Foo</span>
<span class="no">C</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">foo</span> <span class="c1">#=> 2</span>
<span class="no">C</span><span class="p">.</span><span class="nf">freeze</span>
<span class="k">module</span> <span class="nn">Foo</span>
<span class="n">refine</span> <span class="no">C</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">foo</span>
<span class="nb">p</span> <span class="mi">3</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">bar</span> <span class="c1">#=> can't modify frozen class (RuntimeError)</span>
<span class="nb">p</span> <span class="mi">3</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">C</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">foo</span>
<span class="no">C</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">bar</span>
</code></pre>
<pre><code>ruby 2.3.0dev (2015-10-26 trunk 52291) [x86_64-darwin15]
2
test.rb:21:in `block in <module:Foo>': can't modify frozen class (RuntimeError)
from test.rb:17:in `refine'
from test.rb:17:in `<module:Foo>'
from test.rb:16:in `<main>'
</code></pre>
Ruby master - Bug #11613 (Closed): test_aspawn_too_long_path creates too many processes
https://redmine.ruby-lang.org/issues/11613
2015-10-22T16:52:33Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>At least on FreeBSD, spawn("echo|echo|echo|echo|echo|echo|echo| ...20000 times") success and create 20000 zombie processes.<br>
To prevent this you can add rlimit_nproc: 1 because it tests sh itself, don't test spawned echos.</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/test/ruby/test_process.rb b/test/ruby/test_process.rb
index 32dcaed..7877171 100644
</span><span class="gd">--- a/test/ruby/test_process.rb
</span><span class="gi">+++ b/test/ruby/test_process.rb
</span><span class="p">@@ -1600,7 +1600,7 @@</span> class TestProcess < Test::Unit::TestCase
assert_raise(*exs, mesg) do
begin
loop do
<span class="gd">- Process.spawn(cmds.join(sep), [STDOUT, STDERR]=>File::NULL)
</span><span class="gi">+ Process.spawn(cmds.join(sep), [STDOUT, STDERR]=>File::NULL, rlimit_nproc: 1)
</span> min = [cmds.size, min].max
cmds *= 100
end
</code></pre>
Ruby master - Feature #11577 (Open): Add encodeURIComponent compatible API for URI
https://redmine.ruby-lang.org/issues/11577
2015-10-09T13:40:10Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>How about adding encodeURIComponent/decodeURIComponent compatible API?</p>
<p>There's already have some methods:</p>
<ul>
<li>URI.escape: context aware but deprecated.</li>
<li>URIencode_www_form: application/x-www-form-urlencoded, which encodes spaces into '+'</li>
<li>URIencode_www_form_component: above component</li>
</ul>
<p>So it doesn't have non-form URI escape method.</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/lib/uri/common.rb b/lib/uri/common.rb
index 1444ae8..0017ae3 100644
</span><span class="gd">--- a/lib/uri/common.rb
</span><span class="gi">+++ b/lib/uri/common.rb
</span><span class="p">@@ -1,4 +1,5 @@</span>
<span class="gd">-#--
</span><span class="gi">+#
+# -*- frozen-string-literal: true -*-
</span> # = uri/common.rb
#
# Author:: Akira Yamada <akira@ruby-lang.org>
<span class="p">@@ -329,27 +330,72 @@</span> module URI
DEFAULT_PARSER.make_regexp(schemes)
end
<span class="gi">+ TBLENCURICOMP_ = {} # :nodoc:
</span> TBLENCWWWCOMP_ = {} # :nodoc:
<span class="gd">- 256.times do |i|
- TBLENCWWWCOMP_[i.chr] = '%%%02X' % i
- end
- TBLENCWWWCOMP_[' '] = '+'
- TBLENCWWWCOMP_.freeze
</span><span class="gi">+ TBLDECURICOMP_ = {} # :nodoc:
</span> TBLDECWWWCOMP_ = {} # :nodoc:
256.times do |i|
h, l = i>>4, i&15
<span class="gd">- TBLDECWWWCOMP_['%%%X%X' % [h, l]] = i.chr
- TBLDECWWWCOMP_['%%%x%X' % [h, l]] = i.chr
- TBLDECWWWCOMP_['%%%X%x' % [h, l]] = i.chr
- TBLDECWWWCOMP_['%%%x%x' % [h, l]] = i.chr
</span><span class="gi">+ c = i.chr.freeze
+ k = sprintf('%%%X%X', h, l).freeze
+ TBLENCURICOMP_[c] = TBLENCWWWCOMP_[c] = k
+ TBLDECURICOMP_[k] = TBLDECWWWCOMP_[k] = c
+ k = sprintf('%%%x%X', h, l).freeze
+ TBLDECURICOMP_[k] = TBLDECWWWCOMP_[k] = c
+ k = sprintf('%%%X%x', h, l).freeze
+ TBLDECURICOMP_[k] = TBLDECWWWCOMP_[k] = c
+ k = sprintf('%%%x%x', h, l).freeze
+ TBLDECURICOMP_[k] = TBLDECWWWCOMP_[k] = c
</span> end
<span class="gi">+ TBLENCWWWCOMP_[' '] = '+'
+ TBLENCWWWCOMP_.freeze
+ TBLENCURICOMP_.freeze
</span> TBLDECWWWCOMP_['+'] = ' '
TBLDECWWWCOMP_.freeze
<span class="gi">+ TBLDECURICOMP_.freeze
+
+ # Encode given +str+ to URL-encoded form data.
+ #
+ # This method doesn't convert *, -, ., 0-9, A-Z, _, a-z, but does convert SP
+ # (ASCII space) to + and converts others to %XX.
+ #
+ # If +enc+ is given, convert +str+ to the encoding before percent encoding.
+ #
+ # This is an implementation of
+ # http://www.ecma-international.org/ecma-262/6.0/#sec-encodeuricomponent-uricomponent
+ #
+ # See URI.encode_www_form_component, URI.encode_www_form
+ def self.encode_component(str)
+ str = str.to_s.dup
+ if !str.ascii_only?
+ enc = str.encoding
+ if enc != Encoding::ASCII_8BIT && enc != Encoding::UTF_8
+ str.encode!(Encoding::UTF_8, invalid: :replace, undef: :replace)
+ end
+ str.force_encoding(Encoding::ASCII_8BIT)
+ end
+ str.gsub!(/[^\-_.!~*'()0-9A-Za-z]/, TBLENCURICOMP_)
+ str.force_encoding(Encoding::US_ASCII)
+ end
+
+ # Decode given +str+ of URL-encoded form data.
+ #
+ #
+ # This doesn't decodes + to SP.
+ #
+ # This is an implementation of
+ # http://www.ecma-international.org/ecma-262/6.0/#sec-decodeuricomponent-encodeduricomponent
+ #
+ # See URI.decode_www_form_component, URI.decode_www_form
+ def self.decode_component(str, enc=Encoding::UTF_8)
+ raise ArgumentError, "invalid %-encoding (#{str})" if /%(?!\h\h)/ =~ str
+ str.b.gsub(/%\h\h/, TBLDECURICOMP_).force_encoding(enc)
+ end
</span>
HTML5ASCIIINCOMPAT = defined? Encoding::UTF_7 ? [Encoding::UTF_7, Encoding::UTF_16BE, Encoding::UTF_16LE,
Encoding::UTF_32BE, Encoding::UTF_32LE] : [] # :nodoc:
<span class="gd">- # Encode given +str+ to URL-encoded form data.
</span><span class="gi">+ # Encode given +str+ in application/x-www-form-urlencoded format.
</span> #
# This method doesn't convert *, -, ., 0-9, A-Z, _, a-z, but does convert SP
# (ASCII space) to + and converts others to %XX.
<span class="p">@@ -373,7 +419,7 @@</span> module URI
str.force_encoding(Encoding::US_ASCII)
end
<span class="gd">- # Decode given +str+ of URL-encoded form data.
</span><span class="gi">+ # Decode given +str+ in application/x-www-form-urlencoded format.
</span> #
# This decodes + to SP.
#
<span class="p">@@ -457,7 +503,7 @@</span> module URI
if isindex
if sep.empty?
val = key
<span class="gd">- key = ''
</span><span class="gi">+ key = String.new
</span> end
isindex = false
end
<span class="p">@@ -471,7 +517,7 @@</span> module URI
if val
val.gsub!(/\+|%\h\h/, TBLDECWWWCOMP_)
else
<span class="gd">- val = ''
</span><span class="gi">+ val = String.new
</span> end
ary << [key, val]
<span class="gh">diff --git a/test/uri/test_common.rb b/test/uri/test_common.rb
index 5620415..c6b5633 100644
</span><span class="gd">--- a/test/uri/test_common.rb
</span><span class="gi">+++ b/test/uri/test_common.rb
</span><span class="p">@@ -54,6 +54,34 @@</span> class TestCommon < Test::Unit::TestCase
assert_raise(NoMethodError) { Object.new.URI("http://www.ruby-lang.org/") }
end
<span class="gi">+ def test_encode_component
+ assert_equal("%00%20!%22%23%24%25%26'()*%2B%2C-.%2F09%3A%3B%3C%3D%3E%3F%40" \
+ "AZ%5B%5C%5D%5E_%60az%7B%7C%7D~",
+ URI.encode_component("\x00 !\"\#$%&'()*+,-./09:;<=>?@AZ[\\]^_`az{|}~"))
+ assert_equal("%E6%9F%8A", URI.encode_component(
+ "\x95\x41".force_encoding(Encoding::Shift_JIS)))
+ assert_equal("%E3%81%82", URI.encode_component(
+ "\x30\x42".force_encoding(Encoding::UTF_16BE)))
+ assert_equal("%E3%81%82", URI.encode_component(
+ "\e$B$\"\e(B".force_encoding(Encoding::ISO_2022_JP)))
+ end
+
+ def test_decode_component
+ assert_equal(" +!\"\#$%&'()*+,-./09:;<=>?@AZ[\\]^_`az{|}~",
+ URI.decode_component(
+ "%20+%21%22%23%24%25%26%27%28%29*%2B%2C-.%2F09%3A%3B%3C%3D%3E%3F%40" \
+ "AZ%5B%5C%5D%5E_%60az%7B%7C%7D%7E"))
+ assert_equal("\xA1\xA2".force_encoding(Encoding::EUC_JP),
+ URI.decode_component("%A1%A2", "EUC-JP"))
+ assert_equal("\xE3\x81\x82\xE3\x81\x82".force_encoding("UTF-8"),
+ URI.decode_component("\xE3\x81\x82%E3%81%82".force_encoding("UTF-8")))
+
+ assert_raise(ArgumentError){URI.decode_component("%")}
+ assert_raise(ArgumentError){URI.decode_component("%a")}
+ assert_raise(ArgumentError){URI.decode_component("x%a_")}
+ assert_nothing_raised(ArgumentError){URI.decode_component("x"*(1024*1024))}
+ end
+
</span> def test_encode_www_form_component
assert_equal("%00+%21%22%23%24%25%26%27%28%29*%2B%2C-.%2F09%3A%3B%3C%3D%3E%3F%40" \
"AZ%5B%5C%5D%5E_%60az%7B%7C%7D%7E",
</code></pre>
Ruby master - Feature #11558 (Closed): Time related C APIs
https://redmine.ruby-lang.org/issues/11558
2015-09-30T05:59:44Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Time関連のC APIを追加して欲しいです。<br>
具体的には以下のようなものが欲しいです。</p>
<p>struct timespecとoffsetを取って、Timeを返してください。<br>
VALUE rb_time_timespec_new(const struct timespec *ts, int offset);<br>
趣旨としては、[秒, ナノ秒, offset]からTimeを作って欲しいと言うことです。<br>
(rb_time_num_new(Rational, offset)とかだと手間かかる上に遅い)</p>
<p>現在時刻をstruct timespecで欲しいです。<br>
void timespec_now(struct timespec *ts);</p>
<p>既存の非公開APIを公開してください。<br>
struct tm * localtime_with_gmtoff_zone(const time_t *t, struct tm *result, long *gmtoff, const char **zone);<br>
time_t timegm_noleapsecond(struct tm *tm);<br>
void tm_add_offset(struct tm *tm, long diff);</p>
<p><a href="https://github.com/nurse/strptime/blob/v0.1.1/ext/strptime/ruby_time.c" class="external">https://github.com/nurse/strptime/blob/v0.1.1/ext/strptime/ruby_time.c</a></p>
Ruby master - Misc #11516 (Closed): Ruby 2.3.0 release engeneering
https://redmine.ruby-lang.org/issues/11516
2015-09-09T06:13:18Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>For the management of release blockers.</p>
<p>Current Phase:<br>
Next release: preview1</p>
<p>Q: When ruby_2_2 branch is created?<br>
A: On RC1</p>
Ruby master - Misc #11474 (Closed): [META] Call for Feature Proposals for Ruby 2.3
https://redmine.ruby-lang.org/issues/11474
2015-08-21T05:44:04Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>This is meta ticket for Call for Feature Proposals for Ruby 2.3.<br>
After you attach your slide to the related ticket, write ticket number as a comment here to avoid I miss it.</p>
Ruby master - Bug #11457 (Closed): miniruby SEGVs on CentOS 5
https://redmine.ruby-lang.org/issues/11457
2015-08-18T11:03:58Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>minirubyがCentOS 5 (64bit) でSEGVするやつです。<br>
<a href="http://rubyci.s3.amazonaws.com/centos5-64/ruby-trunk/log/20150818T093302Z.log.html.gz#miniversion" class="external">http://rubyci.s3.amazonaws.com/centos5-64/ruby-trunk/log/20150818T093302Z.log.html.gz#miniversion</a></p>
<p>パッチ見ればわかりますが原因はr49452です。<br>
2.1と2.2にはバックポートされているようだけど、2.0.0には入っていないと思う。</p>
Ruby master - Feature #11302 (Closed): Dir.entries and Dir.foreach without [".", ".."]
https://redmine.ruby-lang.org/issues/11302
2015-06-24T06:24:58Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Dir.entries returns an array of its content with "." and "..".<br>
But as far as I met, almost all cases don't need them.</p>
<p>How about adding such new method or options?</p>
Ruby master - Feature #11253 (Closed): rb_io_modestr_oflags for Ruby API
https://redmine.ruby-lang.org/issues/11253
2015-06-12T08:18:12Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>If you have a wrapper of IO.open, you may handle mode.<br>
For example on Windows, you may want to add 'b' / OBINARY to given mode.</p>
<p>But some modes has only integer form like O_CLOEXEC, O_EXCL.<br>
If so you need to convert from modestr to oflags</p>
<p>Therefore how about adding a Ruby API version of rb_io_modestr_oflags</p>
Ruby master - Feature #11251 (Closed): Thread#name and Thread#name=
https://redmine.ruby-lang.org/issues/11251
2015-06-12T02:48:43Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Threadの名前の取得・設定を行うAPIを追加しませんか。</p>
<p><a class="issue tracker-2 status-2 priority-4 priority-default" title="Feature: Thread.new without block. (Assigned)" href="https://redmine.ruby-lang.org/issues/6694">#6694</a> や <a class="issue tracker-2 status-2 priority-4 priority-default" title="Feature: Configuration for Thread/Fiber creation (Assigned)" href="https://redmine.ruby-lang.org/issues/6695">#6695</a> などで断続的に議論がなされ、r47670 で一部自動で名前が付けられるようになったりもしていますが、<br>
Rubyレベルでそれを見る手段は提供されていませんし、明示的に付けることも出来ません。</p>
<p>今でもニーズは結構あるようです。</p>
<blockquote>
<p>"Fluentdでどのスレッドがどのプラグインに属しているのかが簡単に分かるようになるんですよね”<br>
<a href="https://twitter.com/repeatedly/status/608855851131011073" class="external">https://twitter.com/repeatedly/status/608855851131011073</a><br>
"確かにRubyのスレッドに名前付けられたら良さそう。主にログで識別する用途で。"<br>
<a href="https://twitter.com/frsyuki/status/608863065598140417" class="external">https://twitter.com/frsyuki/status/608863065598140417</a></p>
</blockquote>
<pre><code>Linux (glibc 2.12+)
int pthread_setname_np(pthread_t thread, const char *name);
int pthread_getname_np(pthread_t thread, char *name, size_t len);
FreeBSD
void pthread_set_name_np(pthread_t, const char *);
NetBSD
int pthread_getname_np(pthread_t thread, char *name, size_t len);
int pthread_setname_np(pthread_t thread, const char *name, void *arg);
http://netbsd.gw.com/cgi-bin/man-cgi?pthread_setname_np++NetBSD-current
OS X
/usr/include/pthread/pthread.h:int pthread_getname_np(pthread_t,char*,size_t);
/usr/include/pthread/pthread.h:int pthread_setname_np(const char*);
</code></pre>
Ruby master - Feature #11218 (Closed): File.open FILE_SHARE_DELETE
https://redmine.ruby-lang.org/issues/11218
2015-06-04T05:19:41Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>fluentdという、OSSのログコレクタがあるのですが、これには<code>in_tail</code>プラグインというものがあります。<br>
これは、ログファイルを監視して、ログがファイルに追記されたらその分を読み込んでJSONにして他に流します。</p>
<p>このログファイルはfluentdではない他の誰か、ApacheだったりRailsだったりします。<br>
この手のログファイルを書いていく流儀はいくつかありますが、ご存じの通りRailsやApacheは基本的に特定のパスにひたすら追記し続け、<br>
それでは肥大化しすぎるのでlogrotateなどと組み合わせて適宜rotateしていく運用が一般的でしょう。<br>
つまり、ログファイルをrenameし、ApacheやRailsなどにsignalで再起動して新しいファイルに書くわけです。</p>
<p>ところで、Windowsは誰かが開いているファイルをrenameすることはできないので通常このようなことは出来ません。<br>
やるにはファイルを開く際の<code>CreateFile</code>に<code>FILE_SHARE_DELETE</code>フラッグをつけてやる必要があります。<br>
そんなことやってる人いるのかというご指摘もあるとは思いますが、やる人はやっているようです。<br>
<a href="https://issues.apache.org/jira/browse/HADOOP-8564" class="external">https://issues.apache.org/jira/browse/HADOOP-8564</a></p>
<p>さて、RubyのWindowsの<code>open(2)</code>ラッパー、<code>rb_w32_open</code>/<code>rb_w32_wopen</code>はテキストモードの場合は<code>_open</code>/<code>_wopen</code>を使っており、<br>
これはおいておくとして、バイナリモードの場合は<code>CreateFile</code>を使っているものの現状<code>FILE_SHARE_DELETE</code>は渡していないので、<br>
なんらかの手段で<code>FILE_SHARE_DELETE</code>を付けられるようになりませんか。</p>
Ruby master - Feature #11216 (Closed): inode for Windows
https://redmine.ruby-lang.org/issues/11216
2015-06-03T13:32:22Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>現在WindowsではFile::Stat#inodeは常に0を返しますが、<br>
例えばあるlogrotate的な運用の行われるファイルを監視・開きっぱなしにして、追記されれば差分を読み、<br>
rotateされた場合は検出して最新のログを読む…というような場合に、rotateを検出する際にinodeは使われます。</p>
<p>WindowsにはGetFileInformationByHandle()というAPIで得られるBY_HANDLE_FILE_INFORMATION構造体のメンバ、<br>
nFileIndexHigh/Lowから64bitのファイルシステムuniqueなIDがあるので、これをinodeとして代用が可能です。</p>
<p>問題はこれをFile::Statにいれるにはもとのstruct stat.inoが16bitで小さすぎる点ですが、<br>
幸いstruct stati64にはアライメントの都合で隙間があるため、そこにつっこむことにしています。<br>
(devとudevが常に同じとか、uidやgidが常に0など無意味な値は他にもありますが)<br>
<a href="https://msdn.microsoft.com/ja-jp/library/ms350241%28v=vs.71%29.aspx" class="external">https://msdn.microsoft.com/ja-jp/library/ms350241%28v=vs.71%29.aspx</a></p>
<pre><code>diff --git a/file.c b/file.c
index ac99de6..761ff1f 100644
--- a/file.c
+++ b/file.c
@@ -548,7 +548,18 @@ rb_stat_dev_minor(VALUE self)
static VALUE
rb_stat_ino(VALUE self)
{
-#if SIZEOF_STRUCT_STAT_ST_INO > SIZEOF_LONG
+#ifdef _WIN32
+ struct stat *st = get_stat(self);
+ unsigned short *p2 = (unsigned short *)st;
+ unsigned int *p4 = (unsigned int *)st;
+ uint64_t r;
+ r = p2[2];
+ r <<= 16;
+ r |= p2[7];
+ r <<= 32;
+ r |= p4[5];
+ return ULL2NUM(r);
+#elif SIZEOF_STRUCT_STAT_ST_INO > SIZEOF_LONG
return ULL2NUM(get_stat(self)->st_ino);
#else
return ULONG2NUM(get_stat(self)->st_ino);
diff --git a/win32/win32.c b/win32/win32.c
index b23e9af..fcfc0a6 100644
--- a/win32/win32.c
+++ b/win32/win32.c
@@ -4975,6 +4975,48 @@ static time_t filetime_to_unixtime(const FILETIME *ft);
static WCHAR *name_for_stat(WCHAR *buf, const WCHAR *path);
static DWORD stati64_handle(HANDLE h, struct stati64 *st);
+/* License: Ruby's */
+static void
+stati64_set_inode(PBY_HANDLE_FILE_INFORMATION pinfo, struct stati64 *st)
+{
+ /* struct stati64 layout
+ *
+ * dev: 0-3
+ * ino: 4-5
+ * mode: 6-7
+ * nlink: 8-9
+ * uid: 10-11
+ * gid: 12-13
+ * _: 14-15
+ * rdev: 16-19
+ * _: 20-23
+ * size: 24-31
+ * atime: 32-39
+ * mtime: 40-47
+ * ctime: 48-55
+ *
+ */
+ unsigned short *p2 = (unsigned short *)st;
+ unsigned int *p4 = (unsigned int *)st;
+ DWORD high = pinfo->nFileIndexHigh;
+ p2[2] = high >> 16;
+ p2[7] = high & 0xFFFF;
+ p4[5] = pinfo->nFileIndexLow;
+}
+
+/* License: Ruby's */
+static DWORD
+stati64_set_inode_handle(HANDLE h, struct stati64 *st)
+{
+ BY_HANDLE_FILE_INFORMATION info;
+ DWORD attr = (DWORD)-1;
+
+ if (GetFileInformationByHandle(h, &info)) {
+ stati64_set_inode(&info, st);
+ }
+ return attr;
+}
+
#undef fstat
/* License: Ruby's */
int
@@ -5000,7 +5042,11 @@ rb_w32_fstati64(int fd, struct stati64 *st)
struct stat tmp;
int ret;
- if (GetEnvironmentVariableW(L"TZ", NULL, 0) == 0 && GetLastError() == ERROR_ENVVAR_NOT_FOUND) return _fstati64(fd, st);
+ if (GetEnvironmentVariableW(L"TZ", NULL, 0) == 0 && GetLastError() == ERROR_ENVVAR_NOT_FOUND) {
+ ret = _fstati64(fd, st);
+ stati64_set_inode_handle((HANDLE)_get_osfhandle(fd), st);
+ return ret;
+ }
ret = fstat(fd, &tmp);
if (ret) return ret;
@@ -5022,6 +5068,7 @@ stati64_handle(HANDLE h, struct stati64 *st)
st->st_mtime = filetime_to_unixtime(&info.ftLastWriteTime);
st->st_ctime = filetime_to_unixtime(&info.ftCreationTime);
st->st_nlink = info.nNumberOfLinks;
+ stati64_set_inode(&info, st);
attr = info.dwFileAttributes;
}
return attr;
</code></pre>
Ruby master - Bug #11206 (Closed): short file name match incompatibility
https://redmine.ruby-lang.org/issues/11206
2015-06-01T17:54:57Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>When I develop a foo.gem, my working diretory has foo.gemspec and foo.gem.<br>
If I run <code>gem install -l foo</code>, it fails as following:</p>
<p>ERROR: While executing gem ... (Gem::Package::FormatError)<br>
package metadata is missing in foo.gemspec</p>
<p>Because gem command tries to traverse the current directory with Dir["*.gem"],<br>
and it include *.gemspec.</p>
<p>Another example, Rakefile has <code>rake clean</code> task and it is specified by CLEAN constant.<br>
if I specify CLEAN.include( "*.gem" ), it removes *.gemspec.</p>
Ruby master - Bug #11172 (Closed): Windowsでmode: ab+の時に開いたファイルの内容が読めない
https://redmine.ruby-lang.org/issues/11172
2015-05-23T15:23:47Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Windowsでopen("hoge.txt", "ab+")すると、開いたファイルの既存の内容が読めません。<br>
File::APPEND|File::CREAT|File::RDWR|File::BINARYでも同様です。</p>
<p>Unixでは読める上に、a+の時、つまり_openを使う場合も読めるので、意図しない挙動でしょう。</p>
<pre><code>diff --git a/win32/win32.c b/win32/win32.c
index 55e0d2e..57d9df4 100644
--- a/win32/win32.c
+++ b/win32/win32.c
@@ -6476,12 +6476,16 @@ rb_w32_close(int fd)
}
static int
-setup_overlapped(OVERLAPPED *ol, int fd)
+setup_overlapped(OVERLAPPED *ol, int fd, int iswrite)
{
memset(ol, 0, sizeof(*ol));
if (!(_osfile(fd) & (FDEV | FPIPE))) {
LONG high = 0;
- DWORD method = _osfile(fd) & FAPPEND ? FILE_END : FILE_CURRENT;
+ /* On mode:a, it can write only FILE_END.
+ * On mode:a+, though it can write only FILE_END,
+ * it can read from everywhere.
+ */
+ DWORD method = ((_osfile(fd) & FAPPEND) && iswrite) ? FILE_END : FILE_CURRENT;
DWORD low = SetFilePointer((HANDLE)_osfhnd(fd), 0, &high, method);
#ifndef INVALID_SET_FILE_POINTER
#define INVALID_SET_FILE_POINTER ((DWORD)-1)
@@ -6578,7 +6582,7 @@ rb_w32_read(int fd, void *buf, size_t size)
/* if have cancel_io, use Overlapped I/O */
if (cancel_io) {
- if (setup_overlapped(&ol, fd)) {
+ if (setup_overlapped(&ol, fd, FALSE)) {
rb_acrt_lowio_unlock_fh(fd);
return -1;
}
@@ -6708,7 +6712,7 @@ rb_w32_write(int fd, const void *buf, size_t size)
/* if have cancel_io, use Overlapped I/O */
if (cancel_io) {
- if (setup_overlapped(&ol, fd)) {
+ if (setup_overlapped(&ol, fd, TRUE)) {
rb_acrt_lowio_unlock_fh(fd);
return -1;
}
</code></pre>
Ruby master - Misc #10553 (Closed): Ruby 2.2.0 release engeneering
https://redmine.ruby-lang.org/issues/10553
2014-11-28T13:01:41Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>For the management of release blockers.</p>
<p>Current Phase: feature freeze; changing feature is not allowed unless naruse explicitly allowed.<br>
Next release: RC1 (scheduled on 2014-12-1x)</p>
<p>Q: When ruby_2_2 branch is created?<br>
A: On RC1</p>
<p>Q: Can I add/change a feature?<br>
A: Get explicit permission from naruse</p>
Ruby master - Bug #10397 (Closed): gcc 4.1.2 for x86 can't build trunk
https://redmine.ruby-lang.org/issues/10397
2014-10-19T07:12:15Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Recently CentOS 5.6 x86 can't build ruby-trunk.<br>
This is since r47562 and it hit <a href="https://gcc.gnu.org/bugzilla/show_bug.cgi?id=16185" class="external">gcc bug</a>.</p>
<p>Do people want CRuby to continue RHEL/CentOS 5.6 support?<br>
If so, ideally it is fixed by gcc.</p>
Ruby master - Feature #9871 (Open): load a ruby library which doesn't have extension
https://redmine.ruby-lang.org/issues/9871
2014-05-28T10:07:07Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>拡張子のない Ruby スクリプトファイルを require する手段を提供しませんか。</p>
<p>Rubyスクリプトを拡張子無しで書くことはしばしばあります。<br>
例えば Unix のコマンドを Ruby で書くときがそうでしょう。</p>
<p>そのスクリプトの部品を将来再利用しそうなとき、場合によっては if $0 == <strong>FILE</strong> ハックを用いて、<br>
他のファイルから読み込まれたときはコマンドを起動を行わないようにするわけですが、<br>
拡張子がないとそもそも読み込めないので、まずファイル名を変えないといけません。</p>
<p>という具合で残念な感じなので、拡張子無しのファイルを読み込む手段を提供しませんか。<br>
require_relative は拡張子無しでも読める、辺りがいいと思うのですが。</p>
Ruby master - Feature #9816 (Assigned): 文字列内の数字を数値として比較するメソッド
https://redmine.ruby-lang.org/issues/9816
2014-05-08T09:37:26Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>文字列内の数字を数値として比較するメソッドを追加しませんか</p>
<p>そのような比較は一般的な用途としてはGUIシェルのファイラーが比較に用いており、<br>
Windows では StrCmpLogicalW が、OS X では NSString:compare:options:へのNSNumericSearch定数が提供されています。<br>
<a href="http://msdn.microsoft.com/en-us/library/windows/desktop/bb759947(v=vs.85).aspx" class="external">http://msdn.microsoft.com/en-us/library/windows/desktop/bb759947(v=vs.85).aspx</a><br>
<a href="https://developer.apple.com/library/mac/documentation/Cocoa/Reference/Foundation/Classes/NSString_Class/Reference/NSString.html#//apple_ref/c/econst/NSNumericSearch" class="external">https://developer.apple.com/library/mac/documentation/Cocoa/Reference/Foundation/Classes/NSString_Class/Reference/NSString.html#//apple_ref/c/econst/NSNumericSearch</a></p>
<p>上記のような処理自体はさほど難しいものではありませんが、Rubyレベルで実装すると大量のオブジェクトを作ってしまいます。<br>
例えば <code>Gem::Version.new("2.1.10".freeze)<=>Gem::Version.new("2.1.9".freeze)</code> は47個、<br>
<code>"2.1.10".freeze.split('.').map(&:to_i)<=>"2.1.9".freeze.split('.').map(&:to_i)</code> だと16個のオブジェクトを作ります。<br>
<code>"2.1.10".freeze.numericcmp"2.1.9".freeze</code> ならば、もちろんオブジェクトは一つも作りません。</p>
<p>なお、上記の例でも示唆していますが、本メソッドは Ruby のバージョン表記の TEENY が2桁になった場合の比較に用いることができます。</p>
<p>パッチは以下の通りです。<br>
なお、メソッド名は String#numericcmp としています。<br>
(String#casecmpを念頭に置いた)</p>
<pre><code>diff --git a/string.c b/string.c
index c589c80..66f667f 100644
--- a/string.c
+++ b/string.c
@@ -2569,6 +2569,131 @@ rb_str_casecmp(VALUE str1, VALUE str2)
return INT2FIX(-1);
}
+VALUE
+numerical_compare(const char **pp1, const char *p1end, const char **pp2, const char *p2end)
+{
+ const char *s1 = *pp1, *p1, *s2 = *pp2, *p2;
+ ptrdiff_t len1, len2;
+ int r;
+
+ while (s1 < p1end && *s1 == '0') s1++;
+ p1 = s1;
+ while (p1 < p1end && ISDIGIT(*p1)) p1++;
+ len1 = p1 - s1;
+
+ while (s2 < p2end && *s2 == '0') s2++;
+ p2 = s2;
+ while (p2 < p2end && ISDIGIT(*p2)) p2++;
+ len2 = p2 - s2;
+
+ if (len1 != len2) {
+ return INT2FIX(len1 < len2 ? -1 : 1);
+ }
+
+ r = memcmp(s1, s2, len1);
+ if (r) return r < 0 ? INT2FIX(-1) : INT2FIX(1);
+
+ len1 = s1 - *pp1;
+ len2 = s2 - *pp2;
+ if (len1 != len2) {
+ return INT2FIX(len1 < len2 ? -1 : 1);
+ }
+
+ *pp1 = p1;
+ *pp2 = p2;
+ return Qnil;
+}
+
+/*
+ * call-seq:
+ * str.numericcmp(other_str) -> -1, 0, +1 or nil
+ *
+ * Variant of <code>String#<=></code>, which considers digits in strings
+ * are numeric value..
+ *
+ * "a1".numericcmp("a1") #=> 0
+ * "aa".numericcmp("a1") #=> 1
+ * "a1".numericcmp("aa") #=> -1
+ * "a1".numericcmp("a01") #=> -1
+ * "2.1.2".numericcmp("2.1.10") #=> 1
+ */
+
+static VALUE
+rb_str_numericcmp(VALUE str1, VALUE str2)
+{
+ long len;
+ rb_encoding *enc;
+ const char *p1, *p1end, *p2, *p2end;
+
+ StringValue(str2);
+ enc = rb_enc_compatible(str1, str2);
+ if (!enc) {
+ return Qnil;
+ }
+
+ p1 = RSTRING_PTR(str1); p1end = RSTRING_END(str1);
+ p2 = RSTRING_PTR(str2); p2end = RSTRING_END(str2);
+ if (single_byte_optimizable(str1) && single_byte_optimizable(str2)) {
+ while (p1 < p1end && p2 < p2end) {
+ if (ISDIGIT(*p1)) {
+ if (ISDIGIT(*p2)) {
+ VALUE r = numerical_compare(&p1, p1end, &p2, p2end);
+ if (!NIL_P(r)) return r;
+ }
+ else {
+ return INT2FIX(-1);
+ }
+ }
+ else if (ISDIGIT(*p2)) {
+ return INT2FIX(1);
+ }
+ if (*p1 != *p2) return INT2FIX(*p1 < *p2 ? -1 : 1);
+ p1++;
+ p2++;
+ }
+ }
+ else {
+ while (p1 < p1end && p2 < p2end) {
+ int l1, c1 = rb_enc_ascget(p1, p1end, &l1, enc);
+ int l2, c2 = rb_enc_ascget(p2, p2end, &l2, enc);
+
+ if (0 <= c1 && 0 <= c2) {
+ if (ISDIGIT(*p1)) {
+ if (ISDIGIT(*p2)) {
+ VALUE r = numerical_compare(&p1, p1end, &p2, p2end);
+ if (!NIL_P(r)) return r;
+ }
+ else {
+ return INT2FIX(-1);
+ }
+ }
+ else if (ISDIGIT(*p2)) {
+ return INT2FIX(1);
+ }
+ if (*p1 != *p2) return INT2FIX(*p1 < *p2 ? -1 : 1);
+ p1++;
+ p2++;
+ }
+ else {
+ int r;
+ l1 = rb_enc_mbclen(p1, p1end, enc);
+ l2 = rb_enc_mbclen(p2, p2end, enc);
+ len = l1 < l2 ? l1 : l2;
+ r = memcmp(p1, p2, len);
+ if (r != 0)
+ return INT2FIX(r < 0 ? -1 : 1);
+ if (l1 != l2)
+ return INT2FIX(l1 < l2 ? -1 : 1);
+ }
+ p1 += l1;
+ p2 += l2;
+ }
+ }
+ if (RSTRING_LEN(str1) == RSTRING_LEN(str2)) return INT2FIX(0);
+ if (RSTRING_LEN(str1) > RSTRING_LEN(str2)) return INT2FIX(1);
+ return INT2FIX(-1);
+}
+
static long
rb_str_index(VALUE str, VALUE sub, long offset)
{
@@ -8721,6 +8846,7 @@ Init_String(void)
rb_define_method(rb_cString, "eql?", rb_str_eql, 1);
rb_define_method(rb_cString, "hash", rb_str_hash_m, 0);
rb_define_method(rb_cString, "casecmp", rb_str_casecmp, 1);
+ rb_define_method(rb_cString, "numericcmp", rb_str_numericcmp, 1);
rb_define_method(rb_cString, "+", rb_str_plus, 1);
rb_define_method(rb_cString, "*", rb_str_times, 1);
rb_define_method(rb_cString, "%", rb_str_format_m, 1);
diff --git a/test/ruby/test_string.rb b/test/ruby/test_string.rb
index 8366424..f9c788b 100644
--- a/test/ruby/test_string.rb
+++ b/test/ruby/test_string.rb
@@ -2104,6 +2104,29 @@ class TestString < Test::Unit::TestCase
assert_equal(1, "\u3042B".casecmp("\u3042a"))
end
+ def test_numericcmp
+ assert_equal(-1, "2.1.0".numericcmp("2.1.1"))
+ assert_equal(-1, "2.1.9".numericcmp("2.1.10"))
+ assert_equal( 0, "a1".numericcmp("a1"))
+ assert_equal( 1, "aa".numericcmp("a1"))
+ assert_equal(-1, "a1".numericcmp("aa"))
+ assert_equal(-1, "a1".numericcmp("a01"))
+ assert_equal(-1, "a0001".numericcmp("a00001"))
+ assert_equal( 0, "a1a".numericcmp("a1a"))
+ assert_equal( 1, "a1b".numericcmp("a1a"))
+ assert_equal(-1, "a9a".numericcmp("a10a"))
+ assert_equal( 1, "b".numericcmp("a"))
+ assert_equal( 0, "\u30421".numericcmp("\u30421"))
+ assert_equal( 1, "\u3042\u3042".numericcmp("\u30421"))
+ assert_equal(-1, "\u30421".numericcmp("\u3042\u3042"))
+ assert_equal(-1, "\u30421".numericcmp("\u304201"))
+ assert_equal(-1, "\u30420001".numericcmp("\u304200001"))
+ assert_equal( 0, "\u30421\u3042".numericcmp("\u30421\u3042"))
+ assert_equal( 1, "\u30421\u3044".numericcmp("\u30421\u3042"))
+ assert_equal(-1, "\u30429\u3042".numericcmp("\u304210\u3042"))
+ assert_equal( 1, "\u3044".numericcmp("\u3042"))
+ end
+
def test_upcase2
assert_equal("\u3042AB", "\u3042aB".upcase)
end
</code></pre>
Ruby master - Feature #9647 (Closed): File::Stat#birthtimeの追加
https://redmine.ruby-lang.org/issues/9647
2014-03-17T05:13:51Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>File::Stat#birthtimeを追加しませんか。</p>
<p>以下の様なシステムの stat(2) には st_birthtimespec があり、<br>
(ctime = change time ではなく) ファイルを作成した日時を得ることができます。<br>
<a href="http://netbsd.gw.com/cgi-bin/man-cgi?stat+2+NetBSD-current" class="external">http://netbsd.gw.com/cgi-bin/man-cgi?stat+2+NetBSD-current</a><br>
<a href="http://www.freebsd.org/cgi/man.cgi?query=stat&sektion=2&apropos=0&manpath=FreeBSD+10.0-RELEASE" class="external">http://www.freebsd.org/cgi/man.cgi?query=stat&sektion=2&apropos=0&manpath=FreeBSD+10.0-RELEASE</a><br>
<a href="https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/stat.2.html" class="external">https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/stat.2.html</a></p>
<p>また、Windowsはbirthtimeがあるがctimeがないという環境ですので、birthtimeはctimeを返します。<br>
<a href="http://msdn.microsoft.com/ja-jp/library/ms350241(v=vs.71).aspx" class="external">http://msdn.microsoft.com/ja-jp/library/ms350241(v=vs.71).aspx</a></p>
<p>今回のパッチではLinuxやOpenBSDなど、struct statにbirthtimeがない環境では、<br>
Windows同様ctimeを返すようにしています。<br>
(が、意味が違うからWindows以外ではNotImpErrorの方がいいかも)</p>
<p><a href="https://github.com/nurse/ruby/compare/ruby:trunk...birthtime" class="external">https://github.com/nurse/ruby/compare/ruby:trunk...birthtime</a></p>
Ruby master - Bug #9287 (Closed): 'rb_obj_write' discards qualifiers from pointer target type
https://redmine.ruby-lang.org/issues/9287
2013-12-23T18:22:00Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>array.c: In function 'rb_ary_new_from_args':<br>
array.c:506: warning: passing argument 2 of 'rb_obj_write' discards qualifiers from pointer target type<br>
などといった警告が出ます。</p>
<p><a href="http://fb32.rubyci.org/~chkbuild/ruby-trunk/log/20131223T070301Z.log.html.gz" class="external">http://fb32.rubyci.org/~chkbuild/ruby-trunk/log/20131223T070301Z.log.html.gz</a></p>
Ruby master - Feature #8850 (Assigned): Convert Rational to decimal string
https://redmine.ruby-lang.org/issues/8850
2013-09-02T10:02:36Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>On Ruby 2.1.0, decimal literal is introduced.<br>
It generates Rational but it cannot easily convert to decimal string.<br>
You know, Rational#to_f is related to this but</p>
<ul>
<li>
<p>Float is not exact number<br>
** 0.123456789123456789r.to_f.to_s #=> "0.12345678912345678"</p>
</li>
<li>
<p>it can't handle recursive decimal<br>
** (151/13r).to_f.to_s #=> "11.615384615384615"</p>
</li>
<li>
<p>the method name<br>
** to_decimal<br>
** to_decimal_string<br>
** to_s(format: :decimal)<br>
** extend sprintf</p>
</li>
<li>
<p>how does it express recursive decimal<br>
** (151/13r).to_decimal_string #=> "11.615384..."<br>
** (151/13r).to_decimal_string #=> "11.(615384)"</p>
</li>
</ul>
<p>Example implementation is following.<br>
Its result is<br>
** 0.123456789123456789r.to_f.to_s #=> "0.123456789123456789"<br>
** (151/13r).to_f.to_s #=> "11.(615384)"</p>
<pre><code>class Rational
def to_decimal_string(base=10)
n = numerator
d = denominator
r, n = n.divmod d
str = r.to_s(base)
return str if n == 0
h = {}
str << '.'
n *= base
str.size.upto(Float::INFINITY) do |i|
r, n = n.divmod d
if n == 0
str << r.to_s(base)
break
elsif h.key? n
str[h[n], 0] = '('
str << ')'
break
else
str << r.to_s(base)
h[n] = i
n *= base
end
end
str
end
end
</code></pre>
Ruby master - Bug #8782 (Closed): Don't set rl_getc_function on editline
https://redmine.ruby-lang.org/issues/8782
2013-08-12T16:31:55Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>r42402 以来 OS X 等の editline 環境では #define rl_getc(f) EOF が使われるようになってしまって残念なことになっていたわけですが、<br>
そもそも editline の readline wrapper は non ASCII に対応していません。<br>
(editline 自体には UTF-8 のみの対応が入ったが、readline wrapper は src/readline.c の _getc_function を経由するので non ASCII は化ける)</p>
<p>ので、いっそ rl_getc_function を使わないようにしてはどうでしょう。<br>
以下のようなパッチを当てると、readline なしの OS X の irb で日本語が使えるようになります。</p>
<p>diff --git a/ext/readline/extconf.rb b/ext/readline/extconf.rb<br>
index 0b121c1..bc0ee77 100644<br>
--- a/ext/readline/extconf.rb<br>
+++ b/ext/readline/extconf.rb<br>
@@ -94,4 +94,5 @@ readline.have_func("clear_history")<br>
readline.have_func("rl_redisplay")<br>
readline.have_func("rl_insert_text")<br>
readline.have_func("rl_delete_text")<br>
+readline.have_func("el_init")<br>
create_makefile("readline")<br>
diff --git a/ext/readline/readline.c b/ext/readline/readline.c<br>
index 0f76d1a..85109f0 100644<br>
--- a/ext/readline/readline.c<br>
+++ b/ext/readline/readline.c<br>
@@ -130,12 +130,7 @@ static VALUE readline_instream;<br>
static VALUE readline_outstream;</p>
<a name="if-defined-HAVE_RL_GETC_FUNCTION"></a>
<h2 >#if defined HAVE_RL_GETC_FUNCTION<a href="#if-defined-HAVE_RL_GETC_FUNCTION" class="wiki-anchor">¶</a></h2>
<h2>-#ifndef HAVE_RL_GETC<br>
-#define rl_getc(f) EOF<br>
-#endif</h2>
<p>-static int readline_getc(FILE *);<br>
+# ifndef HAVE_EL_INIT<br>
static int<br>
readline_getc(FILE *input)<br>
{<br>
@@ -187,6 +182,7 @@ readline_getc(FILE *input)<br>
#endif<br>
return FIX2INT(c);<br>
}<br>
+# endif<br>
#elif defined HAVE_RL_EVENT_HOOK<br>
#define BUSY_WAIT 0</p>
<p>@@ -1771,7 +1767,9 @@ Init_readline()<br>
/* libedit check rl_getc_function only when rl_initialize() is called, <em>/<br>
/</em> and using_history() call rl_initialize(). <em>/<br>
/</em> This assignment should be placed before using_history() */<br>
+# ifndef HAVE_EL_INIT<br>
rl_getc_function = readline_getc;<br>
+# endif<br>
#elif defined HAVE_RL_EVENT_HOOK<br>
rl_event_hook = readline_event;<br>
#endif</p>
Ruby master - Feature #8734 (Closed): irbに複数行履歴機能が欲しい
https://redmine.ruby-lang.org/issues/8734
2013-08-05T16:22:48Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>現在のirbはクラス定義やメソッド定義のような複数行にわたるものを書くと、<br>
実行結果は閉じるまで返ってこないのに、履歴は一行ごとにバラバラになってしまい、とても不便です。<br>
なので、zshのように、複数行をまとめて履歴として扱えるようになって欲しいです。<br>
参考: <a href="http://news.mynavi.jp/column/zsh/003/index.html" class="external">http://news.mynavi.jp/column/zsh/003/index.html</a></p>
Ruby master - Bug #8711 (Closed): 最近NoMemoryErrorが多い
https://redmine.ruby-lang.org/issues/8711
2013-07-31T16:06:27Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>最近 rubyci で NoMemoryError を出して失敗することが多いので、それを追跡するスレ</p>
<p>= TestFiber#test_many_fibers<br>
<a href="http://u64b.rubyci.org/~chkbuild/ruby-trunk/log/20130730T233301Z.diff.html.gz" class="external">http://u64b.rubyci.org/~chkbuild/ruby-trunk/log/20130730T233301Z.diff.html.gz</a><br>
<a href="http://rbci.lakewood.privs.net/ruby-trunk/log/20130731T001002Z.diff.html.gz" class="external">http://rbci.lakewood.privs.net/ruby-trunk/log/20130731T001002Z.diff.html.gz</a></p>
<p>= FiberError: can't alloc machine stack to fiber<br>
<a href="http://u64.rubyci.org/~chkbuild/ruby-trunk/log/20130729T200302Z.diff.html.gz" class="external">http://u64.rubyci.org/~chkbuild/ruby-trunk/log/20130729T200302Z.diff.html.gz</a></p>
Ruby master - Feature #8678 (Assigned): Allow invalid string to work with regexp
https://redmine.ruby-lang.org/issues/8678
2013-07-24T14:47:22Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Legacy Ruby 1.8 could regexp match with broken strings.<br>
People can find characters from binary data on the age.</p>
<p>After Ruby 1.9, Ruby raises Exception if it does regexp match with broken strings.<br>
So it became hard to work with character-wise regexp matching with binary data.</p>
<p>Following patch allows it with the constant Regexp::LOOSEENCODING.</p>
<p>commit eb0111ff7ae3f563ce201c4a5f724f121336d42d<br>
Author: NARUSE, Yui <a href="mailto:naruse@ruby-lang.org" class="email">naruse@ruby-lang.org</a><br>
Date: Mon Jul 22 05:37:44 2013 +0900</p>
<pre><code>* Regexp
* New constant:
* Regexp::ENCODINGLOOSE: declare execute matching even if the target string
is invalid byte sequence. [experimental]
</code></pre>
<p>diff --git a/NEWS b/NEWS<br>
index f5fe388..ade0b03 100644<br>
--- a/NEWS<br>
+++ b/NEWS<br>
@@ -35,6 +35,11 @@ with all sufficient information, see the ChangeLog file.</p>
<ul>
<li>misc
<ul>
<li>Mutex#owned? is no longer experimental.</li>
</ul>
</li>
</ul>
<p>+* Regexp</p>
<ul>
<li>
<ul>
<li>New constant:</li>
</ul>
</li>
<li>
<ul>
<li>Regexp::ENCODINGLOOSE: declare execute matching even if the target string</li>
</ul>
</li>
<li>
<pre><code> is invalid byte sequence. [experimental]
</code></pre>
</li>
<li>
</ul>
<ul>
<li>String
<ul>
<li>New methods:
<ul>
<li>String#scrub and String#scrub! verify and fix invalid byte sequence.<br>
diff --git a/re.c b/re.c<br>
index e5cc79d..230a2e0 100644<br>
--- a/re.c<br>
+++ b/re.c<br>
@@ -256,6 +256,7 @@ rb_memsearch(const void *x0, long m, const void *y0, long n, rb_encoding *enc)</li>
</ul>
</li>
</ul>
</li>
</ul>
<p>#define REG_LITERAL FL_USER5<br>
#define REG_ENCODING_NONE FL_USER6<br>
+#define REG_ENCODING_LOOSE FL_USER7</p>
<p>#define KCODE_FIXED FL_USER4</p>
<p>@@ -263,6 +264,7 @@ rb_memsearch(const void *x0, long m, const void *y0, long n, rb_encoding *enc)<br>
(ONIG_OPTION_IGNORECASE|ONIG_OPTION_MULTILINE|ONIG_OPTION_EXTEND)<br>
#define ARG_ENCODING_FIXED 16<br>
#define ARG_ENCODING_NONE 32<br>
+#define ARG_ENCODING_LOOSE 64</p>
<p>static int<br>
char_to_option(int c)<br>
@@ -1251,7 +1253,8 @@ rb_reg_prepare_enc(VALUE re, VALUE str, int warn)<br>
{<br>
rb_encoding *enc = 0;</p>
<ul>
<li>if (rb_enc_str_coderange(str) == ENC_CODERANGE_BROKEN) {</li>
</ul>
<ul>
<li>if (!(RBASIC(re)->flags & REG_ENCODING_LOOSE) &&</li>
<li>
<pre><code> rb_enc_str_coderange(str) == ENC_CODERANGE_BROKEN) {
rb_raise(rb_eArgError,
"invalid byte sequence in %s",
rb_enc_name(rb_enc_get(str)));
</code></pre>
</li>
</ul>
<p>@@ -2433,6 +2436,9 @@ rb_reg_initialize(VALUE obj, const char *s, long len, rb_encoding *enc,<br>
if (options & ARG_ENCODING_NONE) {<br>
re->basic.flags |= REG_ENCODING_NONE;<br>
}</p>
<ul>
<li>
<p>if (options & ARG_ENCODING_LOOSE) {</p>
</li>
<li>
<pre><code> re->basic.flags |= REG_ENCODING_LOOSE;
</code></pre>
</li>
<li>
<p>}</p>
<p>re->ptr = make_regexp(RSTRING_PTR(unescaped), RSTRING_LEN(unescaped), enc,<br>
options & ARG_REG_OPTION_MASK, err,<br>
@@ -3091,6 +3097,7 @@ rb_reg_options(VALUE re)<br>
options = RREGEXP(re)->ptr->options & ARG_REG_OPTION_MASK;<br>
if (RBASIC(re)->flags & KCODE_FIXED) options |= ARG_ENCODING_FIXED;<br>
if (RBASIC(re)->flags & REG_ENCODING_NONE) options |= ARG_ENCODING_NONE;</p>
</li>
<li>
<p>if (RBASIC(re)->flags & REG_ENCODING_LOOSE) options |= ARG_ENCODING_LOOSE;<br>
return options;<br>
}</p>
</li>
</ul>
<p>@@ -3579,6 +3586,8 @@ Init_Regexp(void)<br>
rb_define_const(rb_cRegexp, "FIXEDENCODING", INT2FIX(ARG_ENCODING_FIXED));<br>
/* see Regexp.options and Regexp.new */<br>
rb_define_const(rb_cRegexp, "NOENCODING", INT2FIX(ARG_ENCODING_NONE));</p>
<ul>
<li>
<p>/* see Regexp.options and Regexp.new */</p>
</li>
<li>
<p>rb_define_const(rb_cRegexp, "LOOSEENCODING", INT2FIX(ARG_ENCODING_LOOSE));</p>
<p>rb_global_variable(&reg_cache);</p>
</li>
</ul>
<p>diff --git a/string.c b/string.c<br>
index 1d784e3..caf0baf 100644<br>
--- a/string.c<br>
+++ b/string.c<br>
@@ -3970,7 +3970,7 @@ str_gsub(int argc, VALUE *argv, VALUE str, int bang)<br>
cp = sp;<br>
str_enc = STR_ENC_GET(str);<br>
rb_enc_associate(dest, str_enc);</p>
<ul>
<li>ENC_CODERANGE_SET(dest, rb_enc_asciicompat(str_enc) ? ENC_CODERANGE_7BIT : ENC_CODERANGE_VALID);</li>
</ul>
<ul>
<li>
<p>/<em>ENC_CODERANGE_SET(dest, rb_enc_asciicompat(str_enc) ? ENC_CODERANGE_7BIT : ENC_CODERANGE_VALID);</em>/</p>
<p>do {<br>
n++;<br>
diff --git a/test/ruby/test_regexp.rb b/test/ruby/test_regexp.rb<br>
index 11e86ec..b8f6897 100644<br>
--- a/test/ruby/test_regexp.rb<br>
+++ b/test/ruby/test_regexp.rb<br>
@@ -8,6 +8,10 @@ class TestRegexp < Test::Unit::TestCase<br>
$VERBOSE = nil<br>
end</p>
</li>
<li>
<p>def u(str)</p>
</li>
<li>
<p>str.dup.force_encoding(Encoding::UTF_8)</p>
</li>
<li>
<p>end</p>
</li>
<li>
<p>def teardown<br>
$VERBOSE = @verbose<br>
end<br>
@@ -958,6 +962,17 @@ class TestRegexp < Test::Unit::TestCase<br>
}<br>
end</p>
</li>
<li>
<p>def test_encoding_loose</p>
</li>
<li>
<p>str = u("\x80\xE3\x81\x82\x81")</p>
</li>
<li>
<p>assert_equal(0, Regexp.new(".", Regexp::LOOSEENCODING) =~ str)</p>
</li>
<li>
<p>assert_equal(1, Regexp.new(u('\p{Any}'), Regexp::LOOSEENCODING) =~ str)</p>
</li>
<li>
<p>assert_equal(1, Regexp.new("\u3042", Regexp::LOOSEENCODING) =~ str)</p>
</li>
<li>
<p>assert_equal(1, Regexp.new(u('\p{Hiragana}'), Regexp::LOOSEENCODING) =~ str)</p>
</li>
<li>
<p>assert_equal(0, Regexp.new(u('\A.\p{Hiragana}.\z'), Regexp::LOOSEENCODING) =~ str)</p>
</li>
<li>
<p>str = u("\xf1\x80\xE3\x81\x82\x81")</p>
</li>
<li>
<p>assert_equal(0, Regexp.new(u('\A..\p{Hiragana}.\z'), Regexp::LOOSEENCODING) =~ str)</p>
</li>
<li>
<p>end</p>
</li>
<li>
<a name="This-assertion-is-for-porting-x2-tests-in-testpypy-of-Onigmo"></a>
<h1 >This assertion is for porting x2() tests in testpy.py of Onigmo.<a href="#This-assertion-is-for-porting-x2-tests-in-testpypy-of-Onigmo" class="wiki-anchor">¶</a></h1>
<p>def assert_match_at(re, str, positions, msg = nil)<br>
re = Regexp.new(re) unless re.is_a?(Regexp)</p>
</li>
</ul>
Ruby master - Feature #8675 (Closed): Add Readline.point=(pos)
https://redmine.ruby-lang.org/issues/8675
2013-07-24T03:03:13Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>r42146 でのテスト修正で \A を入力しているのですが、rl_point を直接いじった方が正しい気がするので。</p>
<p>diff --git a/ext/readline/readline.c b/ext/readline/readline.c<br>
index 714b08c..03ab724 100644<br>
--- a/ext/readline/readline.c<br>
+++ b/ext/readline/readline.c<br>
@@ -808,6 +808,12 @@ readline_s_get_point(VALUE self)<br>
{<br>
return INT2NUM(rl_point);<br>
}<br>
+static VALUE<br>
+readline_s_set_point(VALUE self, VALUE pos)<br>
+{</p>
<ul>
<li>rl_point = NUM2INT(pos);</li>
<li>return pos;<br>
+}<br>
#else<br>
#define readline_s_get_point rb_f_notimplement<br>
#endif<br>
@@ -1761,6 +1767,8 @@ Init_readline()<br>
readline_s_get_line_buffer, 0);<br>
rb_define_singleton_method(mReadline, "point",<br>
readline_s_get_point, 0);</li>
<li>rb_define_singleton_method(mReadline, "point=",</li>
<li>
<pre><code> readline_s_set_point, 1);
</code></pre>
rb_define_singleton_method(mReadline, "set_screen_size",<br>
readline_s_set_screen_size, 2);<br>
rb_define_singleton_method(mReadline, "get_screen_size",</li>
</ul>
Ruby master - Feature #8539 (Closed): Unbundle ext/tk
https://redmine.ruby-lang.org/issues/8539
2013-06-18T00:25:36Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>How about unbundling ext/tk from Ruby repository?</p>
<p>ext/tk is a bundled extension library for GUI programming with tk.<br>
It is introduced in 1999 and long maintained with CRuby itself.</p>
<p>But nowadays its maintenance is not so active.<br>
Moreover ext/tk is not the de facto standard over Ruby GUI though it is bundled for 14 years.<br>
(maybe because tk is not de facto of GUI toolkit)<br>
GUI libraries for Ruby should compete in the wilds.</p>
<p>So I propose unbundling ext/tk.<br>
It should be another repository for example on github and people should install it as gem.</p>
<p>How do you think?</p>
Ruby master - Feature #8526 (Closed): gemify tk
https://redmine.ruby-lang.org/issues/8526
2013-06-14T10:22:28Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>tk を Ruby のリポジトリから外しませんか。</p>
<p>そもそも tk はサイズベースで言えば、約69MBのRubyリポジトリのうち、8.8MBを占める巨大なライブラリでかつ、<br>
ビルドに際して多くのライブラリを必要とするため、従前より周囲の目が厳しいライブラリでありましたが、<br>
これまでは精力的にメンテされている永井さんに敬意を払って、本格的な削除の提案はされて来ませんでした。</p>
<p>しかし、最近は活動が鈍り、#5199 <a class="issue tracker-4 status-5 priority-4 priority-default closed" title="Backport: Ruby 1.9.3 fails to compile with tcl/tk on s390x (Closed)" href="https://redmine.ruby-lang.org/issues/5465">#5465</a> <a class="issue tracker-1 status-6 priority-4 priority-default closed" title="Bug: compile error for ext/tk/tcltklib.c: ‘ruby_errinfo’ undeclared (Rejected)" href="https://redmine.ruby-lang.org/issues/5686">#5686</a> <a class="issue tracker-1 status-6 priority-4 priority-default closed" title="Bug: Tk::Scrollable の include が成功しない? (Rejected)" href="https://redmine.ruby-lang.org/issues/7000">#7000</a> <a class="issue tracker-1 status-6 priority-4 priority-default closed" title="Bug: Tkで,コマンドにforkを入れると,イベント実行時にクラッシュする (Rejected)" href="https://redmine.ruby-lang.org/issues/7884">#7884</a> <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: "require 'tk'" segfaults on 64-bit linux with Tk 8.6 (Closed)" href="https://redmine.ruby-lang.org/issues/8000">#8000</a> <a class="issue tracker-4 status-5 priority-4 priority-default closed" title="Backport: Tk::Canvas.create raise if type TkcItem (Closed)" href="https://redmine.ruby-lang.org/issues/8319">#8319</a> <a class="issue tracker-1 status-8 priority-4 priority-default closed" title="Bug: Can't build tcl/tk extensions after updating Debian/Ubuntu package (Third Party's Issue)" href="https://redmine.ruby-lang.org/issues/8435">#8435</a> が残タスクとして残っています。<br>
リリースマネージャとしては、このような状況下ではtkをRubyのリポジトリから外していただいて、<br>
gem等で永井さんのペースでメンテしていただくのがよいのではないでしょうか。</p>
Ruby master - Bug #8492 (Closed): ObjectSpace.after_gc_start_hook aborts with GC.stress
https://redmine.ruby-lang.org/issues/8492
2013-06-05T12:45:13Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>以下を実行すると assert(during_gc > 0) に失敗して abort します。<br>
ruby -robjspace -e'ObjectSpace.after_gc_start_hook=proc{};GC.stress=true;{}'<br>
gc.c:3818 の<br>
gc_event_hook(objspace, RUBY_INTERNAL_EVENT_GC_START, 0 /* TODO: pass minor/immediate flag? */);<br>
実行後に、during_gc が 0 になっている模様。</p>
Ruby master - Bug #8484 (Closed): Restoring conditions through the ruby method call during VM pro...
https://redmine.ruby-lang.org/issues/8484
2013-06-04T12:33:15Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>r41041 で、ブロック呼び出し前の VM 内の処理中に Ruby のメソッドが何らかの理由で呼ばれると argv が壊れるという問題を直しました。<br>
具体的には、</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">y</span> <span class="o">=</span> <span class="no">Object</span><span class="p">.</span><span class="nf">new</span>
<span class="k">def</span> <span class="nc">y</span><span class="o">.</span><span class="nf">s</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
<span class="k">yield</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
<span class="k">end</span>
<span class="n">m</span> <span class="o">=</span> <span class="no">Object</span><span class="p">.</span><span class="nf">new</span>
<span class="k">def</span> <span class="nc">m</span><span class="o">.</span><span class="nf">method_missing</span><span class="p">(</span><span class="o">*</span><span class="n">a</span><span class="p">)</span>
<span class="k">super</span>
<span class="k">end</span>
<span class="n">assert_equal</span> <span class="p">[</span><span class="n">m</span><span class="p">,</span> <span class="kp">nil</span><span class="p">],</span> <span class="n">y</span><span class="p">.</span><span class="nf">s</span><span class="p">(</span><span class="n">m</span><span class="p">){</span><span class="o">|</span><span class="n">a</span><span class="p">,</span><span class="n">b</span><span class="o">|</span><span class="p">[</span><span class="n">a</span><span class="p">,</span><span class="n">b</span><span class="p">]}</span>
</code></pre>
<p>のようなコードの場合、</p>
<ul>
<li>
<code>y.s</code> に <code>m</code> と <code>block</code> が渡される</li>
<li>
<code>y.s</code> で <code>yield(a)</code> が呼ばれる</li>
<li>
<code>argv</code>が設定される</li>
<li>
<code>vm_invoke_block</code> 内の <code>VALUE * const rsp = GET_SP() - ci->argc; SET_SP(rsp);</code> で <code>argv</code> の先頭に <code>sp</code> が設定される</li>
<li><code>vm_yield_setup_args</code></li>
<li><code>vm_yield_setup_block_args</code></li>
<li>
<code>y.s</code> のブロックパラメータは2つなのに、引数は一つなので、<code>a.to_ary</code> が呼ばれる (<code>rb_check_array_type(arg0)</code>)</li>
<li>
<code>method_missing</code> が呼ばれる</li>
<li>
<code>super</code> が呼ばれる (このへんで <code>vm _push_frame</code> が呼ばれる)</li>
<li>
<code>vm_push_frame</code>のinitialize local variablesのところで<code>argv</code>に<code>nil</code>が代入されて破壊される</li>
</ul>
<p>で、これ自体は <code>argv</code> を対比しておいて戻せばよいです。<br>
また、<code>SET_SP(rsp)</code>のあたりは、ささださん曰く「 <code>argv</code> に書き込みがない、という前提でそこは作ってるんだよね」だそうな。<br>
しかし、「そんな前提わかるかっ」ですし、<br>
methodの方は <code>SAVE_RESTORE_CI(tmp = rb_check_convert_type(ary, T_ARRAY, "Array", "to_a"), ci);</code> ともうちょっとわかりやすく書いてあるので、<br>
「頂いた問題点を元に、一度全部総点検が必要そうです」とのことですので、お願いします。</p>
Ruby master - Feature #8414 (Closed): String#scrub!
https://redmine.ruby-lang.org/issues/8414
2013-05-17T16:38:09Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>r40390 [Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Replacing ill-formed subsequencce (Closed)" href="https://redmine.ruby-lang.org/issues/6752">#6752</a>] で で追加した String#scrub ですが、破壊的メソッド版も追加していいでしょうか。<br>
lib/uri/common.rb いじっていて、欲しくなったので。</p>
Ruby master - Bug #8408 (Closed): minitest's test may fail randomly
https://redmine.ruby-lang.org/issues/8408
2013-05-15T11:54:58Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>test/minitest/test_minitest_spec.rb may fail as following:<br>
<a href="http://c5664.rubyci.org/~chkbuild/ruby-trunk/log/20130515T013301Z.log.html.gz" class="external">http://c5664.rubyci.org/~chkbuild/ruby-trunk/log/20130515T013301Z.log.html.gz</a></p>
<ol start="9">
<li>
<p>Failure:<br>
TestMeta#test_structure_subclasses [/home/chkbuild/build/20130515T013301Z/ruby/test/minitest/test_minitest_spec.rb:751]:<br>
Expected #<#<a href="Class:0x002abd3bcfd7f8" class="external">Class:0x002abd3bcfd7f8</a>:0x002abd3bf000f0 @<strong>name</strong>=nil, @<strong>io</strong>=nil, @passed=nil> (top-level thingy::inner) to respond to #xyz.</p>
</li>
<li>
<p>Failure:<br>
TestMeta#test_name [/home/chkbuild/build/20130515T013301Z/ruby/test/minitest/test_minitest_spec.rb:668]:<br>
Expected: "ExampleA"<br>
Actual: "top-level thingy::ExampleA"</p>
</li>
<li>
<p>Failure:<br>
TestMeta#test_structure [/home/chkbuild/build/20130515T013301Z/ruby/test/minitest/test_minitest_spec.rb:687]:<br>
--- expected<br>
+++ actual<br>
@@ -1 +1 @@<br>
-"top-level thingy"<br>
+"top-level thingy::top-level thingy"</p>
</li>
<li>
<p>Failure:<br>
TestMeta#test_name2 [/home/chkbuild/build/20130515T013301Z/ruby/test/minitest/test_minitest_spec.rb:680]:<br>
Expected: "ExampleA"<br>
Actual: "top-level thingy::ExampleA"</p>
</li>
</ol>
<p>It seems because of minitest's bug.<br>
minitest's parallelize_me! make tests parallell but its describe method uses single stack.</p>
Ruby master - Misc #8288 (Closed): Ruby 2.1.0 release engeneering
https://redmine.ruby-lang.org/issues/8288
2013-04-19T03:31:49Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>This is meta issue for releasing Ruby 2.1.0</p>
<p>see also <a href="https://bugs.ruby-lang.org/versions/27" class="external">https://bugs.ruby-lang.org/versions/27</a></p>
Backport200 - Backport #8266 (Closed): Backport r40216 (fiddle's mprotect)
https://redmine.ruby-lang.org/issues/8266
2013-04-14T23:23:40Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>r40216 を backport お願いします。<br>
とりあえず segv はしなくなります。</p>
Ruby master - Bug #8252 (Closed): cgiのHTML tag makerに未定義の属性を渡した場合の挙動
https://redmine.ruby-lang.org/issues/8252
2013-04-11T19:58:41Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>r40237 で以下の通り、定義されていない属性を渡したときの挙動が変化しているのですが意図していますか</p>
<ul>
<li>)<br>
+CGI::HtmlExtension#html when passed a Hash returns an 'html'-element using the passed Hash for attributes FAILED<br>
+Expected ""<br>
+to equal "<HTML BLA="TEST">"</li>
</ul>
<p>+/extdisk/chkbuild/chkbuild/tmp/build//rubyspec/library/cgi/htmlextension/html_spec.rb:<line_a>:in <code>block (3 levels) in <top (required)>' +/extdisk/chkbuild/chkbuild/tmp/build/<buildtime>/rubyspec/library/cgi/htmlextension/html_spec.rb:<line_a>:in </code><top (required)>'</p>
<p><a href="http://www.rubyist.net/~akr/chkbuild/debian/ruby-trunk/log/20130411T081100Z.diff.html.gz" class="external">http://www.rubyist.net/~akr/chkbuild/debian/ruby-trunk/log/20130411T081100Z.diff.html.gz</a></p>
Ruby master - Feature #8217 (Closed): OpenSSL::BN.new with integers
https://redmine.ruby-lang.org/issues/8217
2013-04-04T15:59:31Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Current OpenSSL::BN.new gets only strings, so users must do integer.to_s, it costs extra resource.<br>
Therefore I propose OpenSSL::BN.new to allow Fixnu/Bignum.</p>
<p>diff --git a/ext/openssl/ossl_bn.c b/ext/openssl/ossl_bn.c<br>
index 1038135..1f1ebba 100644<br>
--- a/ext/openssl/ossl_bn.c<br>
+++ b/ext/openssl/ossl_bn.c<br>
@@ -120,6 +120,44 @@ ossl_bn_initialize(int argc, VALUE *argv, VALUE self)<br>
base = NUM2INT(bs);<br>
}</p>
<ul>
<li>if (RB_TYPE_P(str, T_FIXNUM)) {</li>
<li>long i;</li>
<li>unsigned char <em>bin = (unsigned char</em>)ALLOC_N(long, 1);</li>
<li>long n = FIX2LONG(str);</li>
<li>unsigned long un = abs(n);</li>
<li>
<li>for (i = sizeof(VALUE) - 1; 0 <= i; i--) {</li>
<li>
<pre><code> bin[i] = un&0xff;
</code></pre>
</li>
<li>
<pre><code> un >>= 8;
</code></pre>
</li>
<li>}</li>
<li>
<li>GetBN(self, bn);</li>
<li>if (!BN_bin2bn(bin, sizeof(long), bn)) {</li>
<li>
<pre><code> ossl_raise(eBNError, NULL);
</code></pre>
</li>
<li>}</li>
<li>if (n < 0) BN_set_negative(bn, 1);</li>
<li>return self;</li>
<li>}</li>
<li>else if (RB_TYPE_P(str, T_BIGNUM)) {</li>
<li>long i, j;</li>
<li>BDIGIT *ds = RBIGNUM_DIGITS(str);</li>
<li>unsigned char <em>bin = (unsigned char</em>)ALLOC_N(BDIGIT, RBIGNUM_LEN(str));</li>
<li>
<li>for (i = 0; RBIGNUM_LEN(str) > i; i++) {</li>
<li>
<pre><code> BDIGIT v = ds[i];
</code></pre>
</li>
<li>
<pre><code> for (j = sizeof(BDIGIT) - 1; 0 <= j; j--) {
</code></pre>
</li>
<li>
<pre><code> bin[(RBIGNUM_LEN(str)-1-i)*sizeof(BDIGIT)+j] = v&0xff;
</code></pre>
</li>
<li>
<pre><code> v >>= 8;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>}</li>
<li>
<li>GetBN(self, bn);</li>
<li>if (!BN_bin2bn(bin, sizeof(BDIGIT)*RBIGNUM_LEN(str), bn)) {</li>
<li>
<pre><code> ossl_raise(eBNError, NULL);
</code></pre>
</li>
<li>}</li>
<li>if (!RBIGNUM_SIGN(str)) BN_set_negative(bn, 1);</li>
<li>return self;</li>
<li>}<br>
if (RTEST(rb_obj_is_kind_of(str, cBN))) {<br>
BIGNUM *other;</li>
</ul>
<p>diff --git a/test/openssl/test_bn.rb b/test/openssl/test_bn.rb<br>
index af1c72c..758cc54 100644<br>
--- a/test/openssl/test_bn.rb<br>
+++ b/test/openssl/test_bn.rb<br>
@@ -8,6 +8,13 @@ class OpenSSL::TestBN < Test::Unit::TestCase<br>
end</p>
<p>def test_integer_to_bn</p>
<ul>
<li>assert_equal(999.to_bn, OpenSSL::BN.new(999))</li>
<li>assert_equal((2 ** 107 - 1).to_bn, OpenSSL::BN.new(2 ** 107 - 1))</li>
<li>assert_equal(-999.to_bn, OpenSSL::BN.new(-999))</li>
<li>assert_equal((-(2 ** 107 - 1)).to_bn, OpenSSL::BN.new(-(2 ** 107 - 1)))</li>
<li>end</li>
<li>
<li>def test_integer_str_to_bn<br>
assert_equal(999.to_bn, OpenSSL::BN.new(999.to_s(16), 16))<br>
assert_equal((2 ** 107 - 1).to_bn, OpenSSL::BN.new((2 ** 107 - 1).to_s(16), 16))<br>
end</li>
</ul>
Ruby master - Bug #8203 (Closed): Rinda: recycled object
https://redmine.ruby-lang.org/issues/8203
2013-04-02T11:00:13Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>test-allで以下のメッセージが出ているんですが、これってなんですか?<br>
<a href="http://u32.rubyci.org/~chkbuild/ruby-trunk/log/20130402T010302Z.log.html.gz#test-all" class="external">http://u32.rubyci.org/~chkbuild/ruby-trunk/log/20130402T010302Z.log.html.gz#test-all</a></p>
<p>Rinda::TupleSpaceProxyTest#test_take_bug_8215 = /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:369:in <code>_id2ref': 0xda91ca76 is recycled object (RangeError) from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:369:in </code>to_obj'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:1446:in <code>to_obj' from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:1748:in </code>to_obj'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:1025:in <code>_load' from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:590:in </code>load'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:590:in <code>block in load' from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:586:in </code>synchronize'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:586:in <code>load' from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:638:in </code>recv_reply'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:936:in <code>recv_reply' from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:1222:in </code>send_message'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:1110:in <code>block (2 levels) in method_missing' from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:1197:in </code>open'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:1109:in <code>block in method_missing' from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:1132:in </code>with_friend'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/drb/drb.rb:1108:in <code>method_missing' from /home/chkbuild/build/20130402T010302Z/ruby/lib/rinda/rinda.rb:265:in </code>write'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/test/rinda/test_rinda.rb:506:in <code>block in test_take_bug_8215' from /home/chkbuild/build/20130402T010302Z/ruby/test/rinda/test_rinda.rb:502:in </code>fork'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/test/rinda/test_rinda.rb:502:in <code>test_take_bug_8215' from /home/chkbuild/build/20130402T010302Z/ruby/lib/test/unit.rb:858:in </code>run_test'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/minitest/unit.rb:1301:in <code>run' from /home/chkbuild/build/20130402T010302Z/ruby/lib/test/unit/testcase.rb:17:in </code>run'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/minitest/unit.rb:919:in <code>block in _run_suite' from /home/chkbuild/build/20130402T010302Z/ruby/lib/minitest/unit.rb:912:in </code>map'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/minitest/unit.rb:912:in <code>_run_suite' from /home/chkbuild/build/20130402T010302Z/ruby/lib/test/unit.rb:657:in </code>block in _run_suites'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/test/unit.rb:655:in <code>each' from /home/chkbuild/build/20130402T010302Z/ruby/lib/test/unit.rb:655:in </code>_run_suites'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/minitest/unit.rb:867:in <code>_run_anything' from /home/chkbuild/build/20130402T010302Z/ruby/lib/minitest/unit.rb:1060:in </code>run_tests'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/minitest/unit.rb:1047:in <code>block in _run' from /home/chkbuild/build/20130402T010302Z/ruby/lib/minitest/unit.rb:1046:in </code>each'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/minitest/unit.rb:1046:in <code>_run' from /home/chkbuild/build/20130402T010302Z/ruby/lib/minitest/unit.rb:1035:in </code>run'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/test/unit.rb:21:in <code>run' from /home/chkbuild/build/20130402T010302Z/ruby/lib/test/unit.rb:774:in </code>run'<br>
from /home/chkbuild/build/20130402T010302Z/ruby/lib/test/unit.rb:834:in <code>run' from /home/chkbuild/build/20130402T010302Z/ruby/lib/test/unit.rb:838:in </code>run'<br>
from ./test/runner.rb:17:in `'<br>
1.42 s = .</p>
Ruby master - Bug #8157 (Closed): How to write document for __LINE__, __FILE__, __END__
https://redmine.ruby-lang.org/issues/8157
2013-03-24T01:12:41Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>How do I write document for <strong>LINE</strong>, <strong>FILE</strong>, <strong>END</strong> ?</p>
<p>Moreover, __ is expressed as _ in rdoc.<br>
How can I avoid it?</p>
Backport193 - Backport #8076 (Closed): Lookbehind assertion fails with /m mode enabled
https://redmine.ruby-lang.org/issues/8076
2013-03-11T19:52:38Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Lookbehind assertions fail if they are longer than one character, and if dotall mode is set.</p>
<p>irb(main):001:0> "foo" =~ /(?<=f).<em>/m<br>
=> 1<br>
irb(main):002:0> "foo" =~ /(?<=fo).</em>/m<br>
=> nil</p>
<p>The latter should have matched the "o". This only seems to happen with dotall mode turned on (dot matches newline); without it, everything is OK:</p>
<p>irb(main):003:0> "foo" =~ /(?<=f).<em>/<br>
=> 1<br>
irb(main):004:0> "foo" =~ /(?<=fo).</em>/<br>
=> 2</p>
Backport193 - Backport #7896 (Closed): Can't test rb_iter_break with extensions
https://redmine.ruby-lang.org/issues/7896
2013-02-21T12:05:29Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>If you test rb_iter_break() with some extension library like trunk's ext/-test-/iter/break.c, it will cause SEGV on 1.9.3.<br>
It won't happen on 2.0 because r34369 is accidentally fix it with below patch.</p>
<p>I noticed this because RubySpec added a test for rb_spec_iter (4db31b04954118e66ac1d6353ebf4106cb2b419b) and hit this.</p>
<h1>% svn di<br>
Index: vm.c</h1>
<p>--- vm.c (revision 39346)<br>
+++ vm.c (working copy)<br>
@@ -1370,6 +1370,7 @@<br>
*th->cfp->sp++ = (GET_THROWOBJ_VAL(err));<br>
#endif<br>
}</p>
<ul>
<li>
<pre><code> th->state = 0;
th->errinfo = Qnil;
goto vm_loop_start;
}
</code></pre>
</li>
</ul>
Ruby master - Bug #7589 (Closed): parallel test-all で test_settracefunc が SEGV
https://redmine.ruby-lang.org/issues/7589
2012-12-19T12:01:03Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>while make TESTS='-qv -j1 ruby/test_settracefunc.rb' test-all;do done としていると以下の通りSEGVします。</p>
<p>前略<br>
TestSetTraceFunc#test_tracepoint_exception_at_line = 0.00 s = .<br>
/home/naruse/ruby/lib/test/unit/parallel.rb:38: [BUG] Segmentation fault<br>
ruby 2.0.0dev (2012-12-19 trunk 38456) [x86_64-linux]</p>
<p>-- Control frame information -----------------------------------------------<br>
c:0002 p:0014 s:0005 e:000004 BLOCK /home/naruse/ruby/lib/test/unit/parallel.rb:38 [FINISH]<br>
c:0001 p:---- s:0002 e:000001 TOP [FINISH]</p>
<p>-- Ruby level backtrace information ----------------------------------------<br>
/home/naruse/ruby/lib/test/unit/parallel.rb:38:in `block in _run_suite'</p>
<p>-- C level backtrace information -------------------------------------------<br>
/home/naruse/ruby/libruby.so.2.0.0(+0x1be874) [0x2b09de81b874] vm_dump.c:643<br>
/home/naruse/ruby/libruby.so.2.0.0(+0x662a7) [0x2b09de6c32a7] error.c:306<br>
/home/naruse/ruby/libruby.so.2.0.0(rb_bug+0x108) [0x2b09de6c33e5] error.c:325<br>
/home/naruse/ruby/libruby.so.2.0.0(+0x1382f2) [0x2b09de7952f2] signal.c:649<br>
/lib/libpthread.so.0(+0xf8f0) [0x2b09deb188f0] ../nptl/sysdeps/pthread/funlockfile.c:30<br>
/home/naruse/ruby/libruby.so.2.0.0(+0x1c13e8) [0x2b09de81e3e8] vm_trace.c:263<br>
/home/naruse/ruby/libruby.so.2.0.0(+0x1c160f) [0x2b09de81e60f] vm_trace.c:309<br>
/home/naruse/ruby/libruby.so.2.0.0(+0x1aba95) [0x2b09de808a95]<br>
/home/naruse/ruby/libruby.so.2.0.0(+0x1baaca) [0x2b09de817aca] vm.c:1169<br>
/home/naruse/ruby/libruby.so.2.0.0(+0x1b9618) [0x2b09de816618] vm.c:636<br>
/home/naruse/ruby/libruby.so.2.0.0(+0x1b9869) [0x2b09de816869] vm.c:684<br>
/home/naruse/ruby/libruby.so.2.0.0(+0x1b9917) [0x2b09de816917] vm.c:703<br>
/home/naruse/ruby/libruby.so.2.0.0(+0x1c56cf) [0x2b09de8226cf]<br>
/home/naruse/ruby/libruby.so.2.0.0(+0x1c4111) [0x2b09de821111] thread_pthread.c:722<br>
/lib/libpthread.so.0(+0x69ca) [0x2b09deb0f9ca] pthread_create.c:300<br>
/lib/libc.so.6(clone+0x6d) [0x2b09df6d921d] parse.y:10551</p>
<p>-- Other runtime information -----------------------------------------------</p>
<ul>
<li>
<p>Loaded script: /home/naruse/ruby/lib/test/unit/parallel.rb: TestSetTraceFunc#test_tracepoint_thread</p>
</li>
<li>
<p>Loaded features:</p>
</li>
</ul>
<p>Some worker was crashed. It seems ruby interpreter's bug<br>
or, a bug of test/unit/parallel.rb. try again without -j<br>
option.</p>
<p>make: *** [yes-test-all] Error 1</p>
<p>以下のパッチを当てると落ちなくなるので、パイプの読み込み部分があやしそうですが……。</p>
<p>diff --git a/lib/test/unit/parallel.rb b/lib/test/unit/parallel.rb<br>
index d189183..b044792 100644<br>
--- a/lib/test/unit/parallel.rb<br>
+++ b/lib/test/unit/parallel.rb<br>
@@ -34,8 +34,13 @@ module Test</p>
<pre><code> th = Thread.new do
begin
</code></pre>
<ul>
<li>
<pre><code> while buf = (self.verbose ? i.gets : i.read(5))
</code></pre>
</li>
<li>
<pre><code> _report "p", buf
</code></pre>
</li>
</ul>
<ul>
<li>
<pre><code> buf = i.read
</code></pre>
</li>
<li>
<pre><code> if self.verbose
</code></pre>
</li>
<li>
<pre><code> buf.each_line{|l|
</code></pre>
</li>
<li>
<pre><code> _report "p", l
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else
</code></pre>
</li>
<li>
<pre><code> raise
end
rescue IOError
rescue Errno::EPIPE
</code></pre>
</li>
</ul>
Backport193 - Backport #7587 (Closed): r35924
https://redmine.ruby-lang.org/issues/7587
2012-12-18T19:56:48Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>make -j したときにこけるので r35924 を backport してください</p>
Ruby master - Bug #7559 (Closed): Wrong line number with method call syntax
https://redmine.ruby-lang.org/issues/7559
2012-12-14T04:06:51Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>以下のような場合に表示される行数がおかしくなります。</p>
<p>def foo(*args)<br>
p caller[0][/\d+/].to_i #=> expected 4 but 8<br>
end<br>
foo "a<br>
b<br>
c<br>
d<br>
e"</p>
<p>def bar(a, b, line)<br>
p a<br>
p line #=> expected 14 but 19<br>
end<br>
def baz<br>
bar <strong>LINE</strong>, <<eom, <strong>LINE</strong><br>
a<br>
b<br>
c<br>
d<br>
eom<br>
end<br>
baz</p>
Ruby master - Bug #7302 (Closed): r37497 changes rb_enumeratorize without NEWS
https://redmine.ruby-lang.org/issues/7302
2012-11-07T19:53:08Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>r37497 [Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Enumerable#size (Closed)" href="https://redmine.ruby-lang.org/issues/6636">#6636</a>] changed the prototype of rb_enumeratorize.<br>
It is public Ruby CAPI.</p>
<p>I think it shouldn't be changed and add another CAPI like rb_enumeratorize_with_size.<br>
At least a description should be add to NEWS</p>
Backport193 - Backport #7169 (Closed): r37169
https://redmine.ruby-lang.org/issues/7169
2012-10-16T09:43:05Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>以下をマージお願いします。</p>
<p>commit 78e3185a20d9307c98fecca6d3bacfb8f2c271b8<br>
Author: naruse <a href="mailto:naruse@b2dd03c8-39d4-4d8f-98ff-823fe69b080e" class="email">naruse@b2dd03c8-39d4-4d8f-98ff-823fe69b080e</a><br>
Date: Sat Oct 13 01:03:41 2012</p>
<pre><code>use tty(1) to check if /dev/tty is usable or not
</code></pre>
<p><a href="http://svn.ruby-lang.org/cgi-bin/viewvc.cgi?revision=37169&view=revision" class="external">http://svn.ruby-lang.org/cgi-bin/viewvc.cgi?revision=37169&view=revision</a></p>
Ruby master - Feature #6936 (Closed): Forbid singleton class and instance variabls for float
https://redmine.ruby-lang.org/issues/6936
2012-08-27T08:12:45Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Introduce Flonum technique to speedup floating computation on th 64bit environment (Closed)" href="https://redmine.ruby-lang.org/issues/6763">#6763</a>] などで議論されていた flonum が r36798 でが入ったわけですが、</p>
<ol>
<li>Float のオブジェクトID の仕様が変更</li>
<li>flonum な float に特異メソッドが追加不可</li>
<li>flonum な float に特異クラスが作成不可</li>
<li>flonum な float は同じ値同士でインスタンス変数が共有される</li>
</ol>
<p>といった非互換が存在します。<br>
もっとも、1. は通常意識するはずのないところですし、2. は元から禁止されています。<br>
気になるのは 3. と 4. で、これは 1.9.3 と挙動が異なるだけでなく、<br>
32bit 環境での 2.0 や、64bit環境の flonum でない float オブジェクトとも挙動が異なります。</p>
<p>実際問題として実害はないような気もしますが、このような違いが極めて実装上の問題で、<br>
Ruby 上から見えないところに存在するのは気持ち悪く感じます。</p>
<p>よって、以下のようにするとよいのではないでしょうか。</p>
<ul>
<li>flonum でない float でも特異クラスの作成を禁止</li>
<li>float へのインスタンス変数作成を禁止</li>
</ul>
<p>後者の具体的手法はいくつかあると思いますが、即値は最初から frozen にしておくとかもありかなと思っています。</p>
<p>話を発散させると、この話は true, false, nil, Fixnum, Symbol のような即値から、<br>
Bignum や Time のような immutable っぽいオブジェクトにも当てはまる気がしています。</p>
Ruby master - Feature #6767 (Closed): Utility method to get a duplicated string whose encoding is...
https://redmine.ruby-lang.org/issues/6767
2012-07-22T03:20:31Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>ある String を ASCII-8BIT にしたいことはしばしばあります。<br>
それだけならばまだ force_encoding しろよという話なのですが、<br>
<a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Bitwise string operations (Rejected)" href="https://redmine.ruby-lang.org/issues/6361">#6361</a> の例のように、バイナリ文字列にさらにバイナリ文字列を結合していく場合、<br>
毎行毎行 force_encoding を書いていくのにはつらいものがあります。</p>
<p>解決案としては、<br>
(1) バイナリリテラルの導入<br>
(2) dup.force_encoding(Encoding::ASCII_8BIT) する短いメソッドを追加<br>
(3) ASCII-8BIT に他のエンコーディングの文字列を結合した場合は暗黙に force_encoding<br>
が考えられます。</p>
<p>しかし、(1) は文法拡張なのでハードルが高く、(3) は方々で議論になっている通りです。<br>
よって、(2) が妥当ではないかと思います。</p>
<p>名前をまつもとさん提案の String#b としたパッチを以下の通り添付します。</p>
<p>diff --git a/string.c b/string.c<br>
index d038835..76cbc36 100644<br>
--- a/string.c<br>
+++ b/string.c<br>
@@ -601,7 +601,7 @@ rb_str_export_to_enc(VALUE str, rb_encoding *enc)<br>
}</p>
<p>static VALUE<br>
-str_replace_shared(VALUE str2, VALUE str)<br>
+str_replace_shared_without_enc(VALUE str2, VALUE str)<br>
{<br>
if (RSTRING_LEN(str) <= RSTRING_EMBED_LEN_MAX) {<br>
STR_SET_EMBED(str2);<br>
@@ -616,8 +616,14 @@ str_replace_shared(VALUE str2, VALUE str)<br>
RSTRING(str2)->as.heap.aux.shared = str;<br>
FL_SET(str2, ELTS_SHARED);<br>
}</p>
<ul>
<li>rb_enc_cr_str_exact_copy(str2, str);</li>
</ul>
<ul>
<li>return str2;<br>
+}</li>
</ul>
<p>+static VALUE<br>
+str_replace_shared(VALUE str2, VALUE str)<br>
+{</p>
<ul>
<li>str_replace_shared_without_enc(str2, str);</li>
<li>rb_enc_cr_str_exact_copy(str2, str);<br>
return str2;<br>
}</li>
</ul>
<p>@@ -7340,6 +7346,23 @@ rb_str_force_encoding(VALUE str, VALUE enc)</p>
<p>/*</p>
<ul>
<li>call-seq:</li>
</ul>
<ul>
<li>
<ul>
<li>
<pre><code>str.b -> str
</code></pre>
</li>
</ul>
</li>
<li>
<ul>
<li>
</ul>
</li>
<li>
<ul>
<li>Returns a copied string whose encoding is ASCII-8BIT.</li>
</ul>
</li>
<li>*/</li>
<li>
</ul>
<p>+static VALUE<br>
+rb_str_b(VALUE str)<br>
+{</p>
<ul>
<li>VALUE str2 = str_alloc(rb_cString);</li>
<li>str_replace_shared_without_enc(str2, str);</li>
<li>OBJ_INFECT(str2, str);</li>
<li>ENC_CODERANGE_SET(str2, ENC_CODERANGE_VALID);</li>
<li>return str2;<br>
+}</li>
<li>
</ul>
<p>+/*</p>
<ul>
<li>
<ul>
<li>
<p>call-seq:</p>
</li>
<li>
<pre><code>str.valid_encoding? -> true or false
</code></pre>
</li>
<li>
<li>
<p>Returns true for a string which encoded correctly.<br>
@@ -7969,6 +7992,7 @@ Init_String(void)</p>
<p>rb_define_method(rb_cString, "encoding", rb_obj_encoding, 0); /* in encoding.c */<br>
rb_define_method(rb_cString, "force_encoding", rb_str_force_encoding, 1);</p>
</li>
</ul>
</li>
<li>rb_define_method(rb_cString, "b", rb_str_b, 0);<br>
rb_define_method(rb_cString, "valid_encoding?", rb_str_valid_encoding_p, 0);<br>
rb_define_method(rb_cString, "ascii_only?", rb_str_is_ascii_only_p, 0);</li>
</ul>
<p>diff --git a/test/ruby/test_m17n.rb b/test/ruby/test_m17n.rb<br>
index dfcaa94..3a4bca7 100644<br>
--- a/test/ruby/test_m17n.rb<br>
+++ b/test/ruby/test_m17n.rb<br>
@@ -1469,4 +1469,14 @@ class TestM17N < Test::Unit::TestCase<br>
yield(*strs)<br>
end<br>
end<br>
+</p>
<ul>
<li>def test_str_b</li>
<li>s = "\u3042"</li>
<li>assert_equal(a("\xE3\x81\x82"), s.b)</li>
<li>assert_equal(Encoding::ASCII_8BIT, s.b.encoding)</li>
<li>s.taint</li>
<li>assert_equal(true, s.b.tainted?)</li>
<li>s.untrust</li>
<li>assert_equal(true, s.b.untrusted?)</li>
<li>end<br>
end</li>
</ul>
Ruby master - Feature #6752 (Closed): Replacing ill-formed subsequencce
https://redmine.ruby-lang.org/issues/6752
2012-07-19T11:42:58Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
== 概要<br>
Stringになんらかの理由で不正なバイト列が含まれている時に、それを置換文字で置き換えたい。</p>
<p>== ユースケース<br>
実際に確認されているユースケースは以下の通りです。</p>
<ul>
<li>
<p>twitterのtitle</p>
</li>
<li>
<p>IRCのログ</p>
</li>
<li>
<p>ニコニコ動画の API</p>
</li>
<li>
<p>Webクローリング<br>
これらの不正なバイト列の生成過程は、おそらく、バイト単位で文字列を切り詰めた時に末尾が切れて、<br>
末尾がおかしい不正な文字列が作られます。(前二者)<br>
これをコンテナに入れたり結合することによって、途中にも混ざった文字列が作られます。(後二者)</p>
</li>
<li>
<p><a href="https://twitter.com/takahashim/status/18974040397" class="external">https://twitter.com/takahashim/status/18974040397</a></p>
</li>
<li>
<p><a href="https://twitter.com/n0kada/status/215674740705210368" class="external">https://twitter.com/n0kada/status/215674740705210368</a></p>
</li>
<li>
<p><a href="https://twitter.com/n0kada/status/215686490070585346" class="external">https://twitter.com/n0kada/status/215686490070585346</a></p>
</li>
<li>
<p><a href="https://twitter.com/hajimehoshi/status/215671146769682432" class="external">https://twitter.com/hajimehoshi/status/215671146769682432</a></p>
</li>
<li>
<p><a href="http://po-ru.com/diary/fixing-invalid-utf-8-in-ruby-revisited/" class="external">http://po-ru.com/diary/fixing-invalid-utf-8-in-ruby-revisited/</a></p>
</li>
<li>
<p><a href="http://stackoverflow.com/questions/2982677/ruby-1-9-invalid-byte-sequence-in-utf-8" class="external">http://stackoverflow.com/questions/2982677/ruby-1-9-invalid-byte-sequence-in-utf-8</a></p>
</li>
</ul>
<p>== 必要な引数: 置換文字<br>
省略可能、String。<br>
デフォルトは、Unicode系ならU+FFFD、それ以外では「?」。<br>
デフォルトが空文字でない理由は、削除してしまうことで、従来は存在しなかったトークンを作れてしまい、<br>
上位のレイヤーの脆弱性に繋がるからです。<br>
<a href="http://unicode.org/reports/tr36/#UTF-8_Exploit" class="external">http://unicode.org/reports/tr36/#UTF-8_Exploit</a></p>
<p>== API<br>
--- str.encode(str.encoding, invalid: replace, [replace: "〓"])</p>
<ul>
<li>CSI的じゃなくて気持ち悪い</li>
<li>iconv でできるのは glibc iconv か GNU libiconv に //IGNORE つけた時で他はできない</li>
<li>実装上のメリットは後述の通り、直感に反してあまりない(と思う)</li>
</ul>
<p>== 別メソッド</p>
<ul>
<li>新しいメソッドである</li>
<li>fix/repair invalid/illegal bytes/sequence あたりの名前か</li>
</ul>
<p>== 実装<br>
=== 鬼車ベース<br>
int ret = rb_enc_precise_mbclen(p, e, enc); して、<br>
MBCLEN_INVALID_P(ret) が真な時、何バイト目が不正なのかわからないのが微妙。<br>
ONIGENC_CONSTRUCT_MBCLEN_INVALID() がバイト数を取らないのが原因なので、<br>
鬼車のエンコーディングモジュール全てに影響してしまうため、修正困難。<br>
不正なバイトはほとんど存在しないと仮定して、効率を犠牲にすれば回避は可能。</p>
<p>=== transcodeベース<br>
UCS正規化なglibc iconv, GNU libiconv, Perl Encodeなどと違って、<br>
CSIなtranscodeでは、自分自身に変換する場合、<br>
エンコーディングごとに「何もしない」変換モジュールを用意しないといけない。</p>
<p>とりあえず鬼車ベースのコンセプト実装とテストを添付しておきます。</p>
<p>diff --git a/string.c b/string.c<br>
index d038835..4808f15 100644<br>
--- a/string.c<br>
+++ b/string.c<br>
@@ -7426,6 +7426,199 @@ rb_str_ellipsize(VALUE str, long len)<br>
return ret;<br>
}</p>
<p>+/*</p>
<ul>
<li>
<ul>
<li>call-seq:</li>
</ul>
</li>
<li>
<ul>
<li>str.fix_invalid -> new_str</li>
</ul>
</li>
<li>
<ul>
<li>
</ul>
</li>
<li>
<ul>
<li>If the string is well-formed, it returns self.</li>
</ul>
</li>
<li>
<ul>
<li>If the string has invalid byte sequence, repair it with given replacement</li>
</ul>
</li>
<li>
<ul>
<li>character.</li>
</ul>
</li>
<li>*/<br>
+VALUE<br>
+rb_str_fix_invalid(VALUE str)<br>
+{</li>
<li>int cr = ENC_CODERANGE(str);</li>
<li>rb_encoding *enc;</li>
<li>if (cr == ENC_CODERANGE_7BIT || cr == ENC_CODERANGE_VALID)</li>
<li>return rb_str_dup(str);</li>
<li>
<li>enc = STR_ENC_GET(str);</li>
<li>if (rb_enc_asciicompat(enc)) {</li>
<li>const char *p = RSTRING_PTR(str);</li>
<li>const char *e = RSTRING_END(str);</li>
<li>const char *p1 = p;</li>
<li>/* 10 should be enough for the usual use case,</li>
<li>
<ul>
<li>fixing a wrongly chopped character at the end of the string</li>
</ul>
</li>
<li>*/</li>
<li>long room = 10;</li>
<li>VALUE buf = rb_str_buf_new(RSTRING_LEN(str) + room);</li>
<li>const char *rep;</li>
<li>if (enc == rb_utf8_encoding())</li>
<li>
<pre><code> rep = "\xEF\xBF\xBD";
</code></pre>
</li>
<li>else</li>
<li>
<pre><code> rep = "?";
</code></pre>
</li>
<li>cr = ENC_CODERANGE_7BIT;</li>
<li>
<li>p = search_nonascii(p, e);</li>
<li>if (!p) {</li>
<li>
<pre><code> p = e;
</code></pre>
</li>
<li>}</li>
<li>while (p < e) {</li>
<li>
<pre><code> int ret = rb_enc_precise_mbclen(p, e, enc);
</code></pre>
</li>
<li>
<pre><code> if (MBCLEN_CHARFOUND_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> if ((unsigned char)*p > 127) cr = ENC_CODERANGE_VALID;
</code></pre>
</li>
<li>
<pre><code> p += MBCLEN_CHARFOUND_LEN(ret);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else if (MBCLEN_INVALID_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> const char *q;
</code></pre>
</li>
<li>
<pre><code> long clen = rb_enc_mbmaxlen(enc);
</code></pre>
</li>
<li>
<pre><code> if (p > p1) rb_str_buf_cat(buf, p1, p - p1);
</code></pre>
</li>
<li>
<pre><code> q = RSTRING_END(buf);
</code></pre>
</li>
<li>
<li>
<pre><code> if (e - p < clen) clen = e - p;
</code></pre>
</li>
<li>
<pre><code> if (clen < 3) {
</code></pre>
</li>
<li>
<pre><code> clen = 1;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else {
</code></pre>
</li>
<li>
<pre><code> long len = RSTRING_LEN(buf);
</code></pre>
</li>
<li>
<pre><code> clen--;
</code></pre>
</li>
<li>
<pre><code> rb_str_buf_cat(buf, p, clen);
</code></pre>
</li>
<li>
<pre><code> for (; clen > 1; clen--) {
</code></pre>
</li>
<li>
<pre><code> ret = rb_enc_precise_mbclen(q, q + clen, enc);
</code></pre>
</li>
<li>
<pre><code> if (MBCLEN_NEEDMORE_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else if (MBCLEN_INVALID_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> continue;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else {
</code></pre>
</li>
<li>
<pre><code> rb_bug("shouldn't reach here '%s'", q);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> rb_str_set_len(buf, len);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> p += clen;
</code></pre>
</li>
<li>
<pre><code> p1 = p;
</code></pre>
</li>
<li>
<pre><code> rb_str_buf_cat2(buf, rep);
</code></pre>
</li>
<li>
<pre><code> p = search_nonascii(p, e);
</code></pre>
</li>
<li>
<pre><code> if (!p) {
</code></pre>
</li>
<li>
<pre><code> p = e;
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else if (MBCLEN_NEEDMORE_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else {
</code></pre>
</li>
<li>
<pre><code> rb_bug("shouldn't reach here");
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>}</li>
<li>if (p1 < p) {</li>
<li>
<pre><code> rb_str_buf_cat(buf, p1, p - p1);
</code></pre>
</li>
<li>}</li>
<li>if (p < e) {</li>
<li>
<pre><code> rb_str_buf_cat2(buf, rep);
</code></pre>
</li>
<li>
<pre><code> cr = ENC_CODERANGE_VALID;
</code></pre>
</li>
<li>}</li>
<li>ENCODING_CODERANGE_SET(buf, rb_enc_to_index(enc), cr);</li>
<li>return buf;</li>
<li>}</li>
<li>else if (rb_enc_dummy_p(enc)) {</li>
<li>return rb_str_dup(str);</li>
<li>}</li>
<li>else {</li>
<li>/* ASCII incompatible */</li>
<li>const char *p = RSTRING_PTR(str);</li>
<li>const char *e = RSTRING_END(str);</li>
<li>const char *p1 = p;</li>
<li>/* 10 should be enough for the usual use case,</li>
<li>
<ul>
<li>fixing a wrongly chopped character at the end of the string</li>
</ul>
</li>
<li>*/</li>
<li>long room = 10;</li>
<li>VALUE buf = rb_str_buf_new(RSTRING_LEN(str) + room);</li>
<li>const char *rep;</li>
<li>long mbminlen = rb_enc_mbminlen(enc);</li>
<li>static rb_encoding *utf16be;</li>
<li>static rb_encoding *utf16le;</li>
<li>static rb_encoding *utf32be;</li>
<li>static rb_encoding *utf32le;</li>
<li>if (!utf16be) {</li>
<li>
<pre><code> utf16be = rb_enc_find("UTF-16BE");
</code></pre>
</li>
<li>
<pre><code> utf16le = rb_enc_find("UTF-16LE");
</code></pre>
</li>
<li>
<pre><code> utf32be = rb_enc_find("UTF-32BE");
</code></pre>
</li>
<li>
<pre><code> utf32le = rb_enc_find("UTF-32LE");
</code></pre>
</li>
<li>}</li>
<li>if (enc == utf16be) {</li>
<li>
<pre><code> rep = "\xFF\xFD";
</code></pre>
</li>
<li>}</li>
<li>else if (enc == utf16le) {</li>
<li>
<pre><code> rep = "\xFD\xFF";
</code></pre>
</li>
<li>}</li>
<li>else if (enc == utf32be) {</li>
<li>
<pre><code> rep = "\x00\x00\xFF\xFD";
</code></pre>
</li>
<li>}</li>
<li>else if (enc == utf32le) {</li>
<li>
<pre><code> rep = "\xFD\xFF\x00\x00";
</code></pre>
</li>
<li>}</li>
<li>else {</li>
<li>
<pre><code> rep = "?";
</code></pre>
</li>
<li>}</li>
<li>
<li>while (p < e) {</li>
<li>
<pre><code> int ret = rb_enc_precise_mbclen(p, e, enc);
</code></pre>
</li>
<li>
<pre><code> if (MBCLEN_CHARFOUND_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> p += MBCLEN_CHARFOUND_LEN(ret);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else if (MBCLEN_INVALID_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> const char *q;
</code></pre>
</li>
<li>
<pre><code> long clen = rb_enc_mbmaxlen(enc);
</code></pre>
</li>
<li>
<pre><code> if (p > p1) rb_str_buf_cat(buf, p1, p - p1);
</code></pre>
</li>
<li>
<pre><code> q = RSTRING_END(buf);
</code></pre>
</li>
<li>
<li>
<pre><code> if (e - p < clen) clen = e - p;
</code></pre>
</li>
<li>
<pre><code> if (clen < mbminlen * 3) {
</code></pre>
</li>
<li>
<pre><code> clen = mbminlen;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else {
</code></pre>
</li>
<li>
<pre><code> long len = RSTRING_LEN(buf);
</code></pre>
</li>
<li>
<pre><code> clen -= mbminlen;
</code></pre>
</li>
<li>
<pre><code> rb_str_buf_cat(buf, p, clen);
</code></pre>
</li>
<li>
<pre><code> for (; clen > mbminlen; clen-=mbminlen) {
</code></pre>
</li>
<li>
<pre><code> ret = rb_enc_precise_mbclen(q, q + clen, enc);
</code></pre>
</li>
<li>
<pre><code> if (MBCLEN_NEEDMORE_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else if (MBCLEN_INVALID_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> continue;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else {
</code></pre>
</li>
<li>
<pre><code> rb_bug("shouldn't reach here '%s'", q);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> rb_str_set_len(buf, len);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> p += clen;
</code></pre>
</li>
<li>
<pre><code> p1 = p;
</code></pre>
</li>
<li>
<pre><code> rb_str_buf_cat2(buf, rep);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else if (MBCLEN_NEEDMORE_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else {
</code></pre>
</li>
<li>
<pre><code> rb_bug("shouldn't reach here");
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>}</li>
<li>if (p1 < p) {</li>
<li>
<pre><code> rb_str_buf_cat(buf, p1, p - p1);
</code></pre>
</li>
<li>}</li>
<li>if (p < e) {</li>
<li>
<pre><code> rb_str_buf_cat2(buf, rep);
</code></pre>
</li>
<li>}</li>
<li>ENCODING_CODERANGE_SET(buf, rb_enc_to_index(enc), ENC_CODERANGE_VALID);</li>
<li>return buf;</li>
<li>}<br>
+}</li>
<li>
</ul>
<p>/**********************************************************************</p>
<ul>
<li>Document-class: Symbol</li>
<li>
</ul>
<p>@@ -7882,6 +8075,7 @@ Init_String(void)<br>
rb_define_method(rb_cString, "getbyte", rb_str_getbyte, 1);<br>
rb_define_method(rb_cString, "setbyte", rb_str_setbyte, 2);<br>
rb_define_method(rb_cString, "byteslice", rb_str_byteslice, -1);</p>
<ul>
<li>
<p>rb_define_method(rb_cString, "fix_invalid", rb_str_fix_invalid, 0);</p>
<p>rb_define_method(rb_cString, "to_i", rb_str_to_i, -1);<br>
rb_define_method(rb_cString, "to_f", rb_str_to_f, 0);<br>
diff --git a/test/ruby/test_string.rb b/test/ruby/test_string.rb<br>
index 47f349c..2b0cfeb 100644<br>
--- a/test/ruby/test_string.rb<br>
+++ b/test/ruby/test_string.rb<br>
@@ -2031,6 +2031,29 @@ class TestString < Test::Unit::TestCase</p>
<p>assert_equal(u("\x82")+("\u3042"*9), ("\u3042"*10).byteslice(2, 28))<br>
end</p>
</li>
<li>
<li>
<p>def test_fix_invalid</p>
</li>
<li>
<p>assert_equal("\uFFFD\uFFFD\uFFFD", "\x80\x80\x80".fix_invalid)</p>
</li>
<li>
<p>assert_equal("\uFFFDA", "\xF4\x80\x80A".fix_invalid)</p>
</li>
<li>
<li>
<a name="exapmles-in-Unicode-610-D93b"></a>
<h1 >exapmles in Unicode 6.1.0 D93b<a href="#exapmles-in-Unicode-610-D93b" class="wiki-anchor">¶</a></h1>
</li>
<li>
<p>assert_equal("\x41\uFFFD\uFFFD\x41\uFFFD\x41",</p>
</li>
<li>
<pre><code> "\x41\xC0\xAF\x41\xF4\x80\x80\x41".fix_invalid)
</code></pre>
</li>
<li>
<p>assert_equal("\x41\uFFFD\uFFFD\uFFFD\x41",</p>
</li>
<li>
<pre><code> "\x41\xE0\x9F\x80\x41".fix_invalid)
</code></pre>
</li>
<li>
<p>assert_equal("\u0061\uFFFD\uFFFD\uFFFD\u0062\uFFFD\u0063\uFFFD\uFFFD\u0064",</p>
</li>
<li>
<pre><code> "\x61\xF1\x80\x80\xE1\x80\xC2\x62\x80\x63\x80\xBF\x64".fix_invalid)
</code></pre>
</li>
<li>
<li>
<p>assert_equal("abcdefghijklmnopqrstuvwxyz\u0061\uFFFD\uFFFD\uFFFD\u0062\uFFFD\u0063\uFFFD\uFFFD\u0064",</p>
</li>
<li>
<pre><code> "abcdefghijklmnopqrstuvwxyz\x61\xF1\x80\x80\xE1\x80\xC2\x62\x80\x63\x80\xBF\x64".fix_invalid)
</code></pre>
</li>
<li>
<li>
<p>assert_equal("\uFFFD\u3042".encode("UTF-16BE"),</p>
</li>
<li>
<pre><code> "\xD8\x00\x30\x42".force_encoding(Encoding::UTF_16BE).
</code></pre>
</li>
<li>
<pre><code> fix_invalid)
</code></pre>
</li>
<li>
<p>assert_equal("\uFFFD\u3042".encode("UTF-16LE"),</p>
</li>
<li>
<pre><code> "\x00\xD8\x42\x30".force_encoding(Encoding::UTF_16LE).
</code></pre>
</li>
<li>
<pre><code> fix_invalid)
</code></pre>
</li>
<li>
<p>end<br>
end</p>
</li>
</ul>
<p>class TestString2 < TestString<br>
=end</p>
Ruby master - Bug #6577 (Closed): GC中にstack overflowが発生するとSEGVする
https://redmine.ruby-lang.org/issues/6577
2012-06-12T02:18:44Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>GC中にstack overflowが発生すると、例外作成時にrb_new_objするので[BUG]ります。</p>
<p>原因<br>
(1) caller のテストで Fiber を利用するようにした<br>
(2) caller 実行中に GC が発生<br>
(3) GC 中にマシンスタックオーバーフロー(SEGV)が発生<br>
(4) スタックオーバーフローエラーを作成<br>
(5) スタックオーバーフローエラーを作るときに object allocation している<br>
(6) -> [BUG]</p>
<p>対処法:<br>
スタックオーバーフローエラーを投げるときはオブジェクト作らないようにする<br>
対症療法:<br>
caller のテストで Fiber を使わないようにする<br>
対症療法その2:<br>
callerのテストで GC.disable</p>
<p>nariさんがGC で再帰しないようにするなんて構想も先日語っておられましたが。</p>
Ruby master - Bug #6556 (Closed): ネストした配列の inspect で segv
https://redmine.ruby-lang.org/issues/6556
2012-06-08T00:36:42Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>./miniruby -e'10000.times.inject(x=[]){|a,|a<<(b=[]);b};x.inspect'<br>
で segv します。</p>
<p>% ./miniruby -e'10000.times.inject(x=[]){|a,|a<<(b=[]);b};x.inspect'|&less<br>
-e:1: [BUG] Segmentation fault<br>
ruby 2.0.0dev (2012-06-06 trunk 35950) [x86_64-freebsd9.0]</p>
<p>-- Control frame information -----------------------------------------------<br>
c:3116 p:---- s:6234 b:6234 l:006233 d:006233 CFUNC :inspect<br>
c:3115 p:---- s:6232 b:6232 l:006231 d:006231 CFUNC :inspect<br>
c:3114 p:---- s:6230 b:6230 l:006229 d:006229 CFUNC :inspect<br>
(中略)<br>
c:0005 p:---- s:0012 b:0012 l:000011 d:000011 CFUNC :inspect<br>
c:0004 p:---- s:0010 b:0010 l:000009 d:000009 CFUNC :inspect<br>
c:0003 p:0041 s:0007 b:0007 l:001458 d:002430 EVAL -e:1<br>
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH<br>
c:0001 p:0000 s:0002 b:0002 l:001458 d:001458 TOP</p>
<p>-e:1:in <code><main>' -e:1:in </code>inspect'<br>
-e:1:in <code>inspect' (中略) -e:1:in </code>inspect'<br>
-e:1:in `inspect'</p>
<p>-- C level backtrace information -------------------------------------------<br>
0x44c8bd <rb_warning+734> at /home/naruse/obj/ruby/miniruby ../../ruby/error.c:269<br>
0x44c9d8 <rb_bug+228> at /home/naruse/obj/ruby/miniruby ../../ruby/error.c:288<br>
0x518181 <ruby_posix_signal+352> at /home/naruse/obj/ruby/miniruby ../../ruby/signal.c:577<br>
0x800ca7723 <_pthread_sigmask+707> at /lib/libthr.so.3<br>
0x800ca7897 <_pthread_sigmask+1079> at /lib/libthr.so.3<br>
0x7ffffffff003</p>
<p>-- Other runtime information -----------------------------------------------</p>
<ul>
<li>
<p>Loaded script: -e</p>
</li>
<li>
<p>Loaded features:</p>
<p>0 enumerator.so</p>
</li>
</ul>
<p>[NOTE]<br>
You may have encountered a bug in the Ruby interpreter or extension libraries.<br>
Bug reports are welcome.<br>
For details: <a href="http://www.ruby-lang.org/bugreport.html" class="external">http://www.ruby-lang.org/bugreport.html</a></p>
<p>zsh: abort (core dumped) ./miniruby -e'p 10000.times.inject(x=[]){|a,|a<<(b=[]);b};x.inspect'</p>
Ruby master - Bug #6405 (Closed): Re: [ruby-cvs:42717] ryan:r35541 (trunk): Imported minitest 2....
https://redmine.ruby-lang.org/issues/6405
2012-05-05T17:27:47Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>After r35541, test-all fails as following:</p>
<ol start="2">
<li>
<p>Error:<br>
test_equals_tilde(TestGemPlatform):<br>
TypeError: can't convert Gem::Platform to String<br>
/home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/test/rubygems/test_gem_platform.rb:210:in <code>test_equals_tilde' /home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/lib/test/unit.rb:633:in </code>block in _run_suites'<br>
/home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/lib/test/unit.rb:631:in <code>each' /home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/lib/test/unit.rb:631:in </code>_run_suites'<br>
/home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/lib/test/unit.rb:21:in <code>run' /home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/lib/test/unit.rb:682:in </code>run'<br>
/home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/lib/test/unit.rb:714:in <code>run' /home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/lib/test/unit.rb:718:in </code>run'<br>
./test/runner.rb:15:in `'</p>
</li>
<li>
<p>Error:<br>
test_dir(TestGemInstaller):<br>
TypeError: can't convert Regexp to String<br>
/home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/test/rubygems/test_gem_installer.rb:1220:in <code>test_dir' /home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/lib/test/unit.rb:633:in </code>block in _run_suites'<br>
/home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/lib/test/unit.rb:631:in <code>each' /home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/lib/test/unit.rb:631:in </code>_run_suites'<br>
/home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/lib/test/unit.rb:21:in <code>run' /home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/lib/test/unit.rb:682:in </code>run'<br>
/home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/lib/test/unit.rb:714:in <code>run' /home/chkbuild/build/ruby-trunk/20120504T230101Z/ruby/lib/test/unit.rb:718:in </code>run'<br>
./test/runner.rb:15:in `'</p>
</li>
</ol>
<p>Why don't you run tests before commit.</p>
<p>(2012/05/05 6:46), <a href="mailto:ryan@ruby-lang.org" class="email">ryan@ruby-lang.org</a> wrote:</p>
<blockquote>
<p>ryan 2012-05-05 06:46:01 +0900 (Sat, 05 May 2012)</p>
<p>New Revision: 35541</p>
<p><a href="http://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=rev&revision=35541" class="external">http://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=rev&revision=35541</a></p>
<p>Log:<br>
Imported minitest 2.12.1 (r7323)</p>
<p>Added files:<br>
trunk/test/minitest/metametameta.rb<br>
Modified files:<br>
trunk/ChangeLog<br>
trunk/lib/minitest/README.txt<br>
trunk/lib/minitest/autorun.rb<br>
trunk/lib/minitest/benchmark.rb<br>
trunk/lib/minitest/mock.rb<br>
trunk/lib/minitest/pride.rb<br>
trunk/lib/minitest/spec.rb<br>
trunk/lib/minitest/unit.rb<br>
trunk/test/minitest/test_minitest_benchmark.rb<br>
trunk/test/minitest/test_minitest_mock.rb<br>
trunk/test/minitest/test_minitest_spec.rb<br>
trunk/test/minitest/test_minitest_unit.rb</p>
</blockquote>
<p>--<br>
NARUSE, Yui <a href="mailto:naruse@airemix.jp" class="email">naruse@airemix.jp</a></p>
Ruby master - Bug #6400 (Closed): dl/callback with fiddle occurs SEGV on NetBSD amd64
https://redmine.ruby-lang.org/issues/6400
2012-05-04T21:33:37Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>On NetBSD amd64, libffi with callback occurs SEGV as following.</p>
<p>kelvena% cat p<br>
require 'dl/callback'<br>
require 'dl/func'<br>
include DL<br>
Called_with = nil<br>
addr = set_callback(TYPE_VOID, 1) do |str|<br>
called_with = dlunwrap(str)<br>
end<br>
func = CFunc.new(addr, TYPE_VOID, 'test')<br>
f = Function.new(func, [TYPE_VOIDP])<br>
arg = 'foo'<br>
f.call(dlwrap(arg))<br>
kelvena% ./ruby p<br>
/home/naruse/local/ruby/lib/ruby/2.0.0/dl/func.rb:55: [BUG] Segmentation fault<br>
ruby 2.0.0dev (2012-04-30 trunk 35500) [x86_64-netbsd6.99.5]</p>
<p>-- Control frame information -----------------------------------------------<br>
c:0005 p:---- s:0022 b:0022 l:000021 d:000021 CFUNC :call<br>
c:0004 p:0059 s:0018 b:0018 l:000017 d:000017 METHOD /home/naruse/local/ruby/lib/ruby/2.0.0/dl/func.<br>
rb:55<br>
c:0003 p:0157 s:0010 b:0010 l:001db8 d:0021a8 EVAL p:11<br>
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH<br>
c:0001 p:0000 s:0002 b:0002 l:001db8 d:001db8 TOP</p>
<p>-- Ruby level backtrace information ----------------------------------------<br>
p:11:in <code><main>' /home/naruse/local/ruby/lib/ruby/2.0.0/dl/func.rb:55:in </code>call'<br>
/home/naruse/local/ruby/lib/ruby/2.0.0/dl/func.rb:55:in `call'</p>
<p>-- Other runtime information -----------------------------------------------</p>
<ul>
<li>
<p>Loaded script: p</p>
</li>
<li>
<p>Loaded features:</p>
<p>0 enumerator.so<br>
1 /home/naruse/local/ruby/lib/ruby/2.0.0/x86_64-netbsd6.99.5/enc/encdb.so<br>
2 /home/naruse/local/ruby/lib/ruby/2.0.0/x86_64-netbsd6.99.5/enc/trans/transdb.so<br>
3 /home/naruse/local/ruby/lib/ruby/2.0.0/rubygems/defaults.rb<br>
4 /home/naruse/local/ruby/lib/ruby/2.0.0/x86_64-netbsd6.99.5/rbconfig.rb<br>
5 /home/naruse/local/ruby/lib/ruby/2.0.0/rubygems/deprecate.rb<br>
6 /home/naruse/local/ruby/lib/ruby/2.0.0/rubygems/exceptions.rb<br>
7 /home/naruse/local/ruby/lib/ruby/2.0.0/rubygems/custom_require.rb<br>
8 /home/naruse/local/ruby/lib/ruby/2.0.0/rubygems.rb<br>
9 /home/naruse/local/ruby/lib/ruby/2.0.0/x86_64-netbsd6.99.5/dl.so<br>
10 /home/naruse/local/ruby/lib/ruby/2.0.0/x86_64-netbsd6.99.5/fiddle.so<br>
11 /home/naruse/local/ruby/lib/ruby/2.0.0/fiddle/function.rb<br>
12 /home/naruse/local/ruby/lib/ruby/2.0.0/fiddle/closure.rb<br>
13 /home/naruse/local/ruby/lib/ruby/2.0.0/fiddle.rb<br>
14 /home/naruse/local/ruby/lib/ruby/2.0.0/dl.rb<br>
15 /home/naruse/local/ruby/lib/ruby/2.0.0/thread.rb<br>
16 /home/naruse/local/ruby/lib/ruby/2.0.0/dl/callback.rb<br>
17 /home/naruse/local/ruby/lib/ruby/2.0.0/dl/stack.rb<br>
18 /home/naruse/local/ruby/lib/ruby/2.0.0/dl/value.rb<br>
19 /home/naruse/local/ruby/lib/ruby/2.0.0/dl/func.rb</p>
</li>
</ul>
<p>[NOTE]<br>
You may have encountered a bug in the Ruby interpreter or extension libraries.<br>
Bug reports are welcome.<br>
For details: <a href="http://www.ruby-lang.org/bugreport.html" class="external">http://www.ruby-lang.org/bugreport.html</a></p>
<p>zsh: abort (core dumped) ./ruby p</p>
Ruby master - Bug #6272 (Closed): Rinda sticks on some tests
https://redmine.ruby-lang.org/issues/6272
2012-04-08T22:27:08Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Rindaが例えば以下のようにすると刺さります。</p>
<p>while [ yes ];do;make test-all TESTS='-v -n test_ruby_talk_264062 rinda/test_rinda.rb' RUBYOPT=-w;done</p>
<p>しばらく追ってみたところ、Rinda は Monitor#synchronize をすることで同期を守っているのですが、<br>
そのテストでは時刻を Rinda::MockClock で扱い、その中で Rinda::MockClock::MyTS を用いて時刻を配信しているのですが、<br>
この TupleSpace は複数のスレッドから触られるため、同一のスレッドからなら何度入ってもロックしない<br>
Monitor#synchronize でも、デッドロックしてしまうからっぽい気がします。</p>
Ruby master - Bug #5868 (Closed): make failed on i686-linux from Bitmap Marking GC
https://redmine.ruby-lang.org/issues/5868
2012-01-09T10:54:12Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>i686-linux に置いて、r34225 以降 make に失敗します。</p>
<p><a href="http://c5632.rubyci.org/~chkbuild/ruby-trunk/log/20120109T010103Z.log.html.gz" class="external">http://c5632.rubyci.org/~chkbuild/ruby-trunk/log/20120109T010103Z.log.html.gz</a><br>
<a href="http://u32.rubyci.org/~chkbuild/ruby-trunk/log/20120108T230102Z.log.html.gz" class="external">http://u32.rubyci.org/~chkbuild/ruby-trunk/log/20120108T230102Z.log.html.gz</a><br>
<a href="http://www.rubyist.net/~akr/chkbuild/debian/ruby-trunk/log/20120109T000400Z.log.html.gz" class="external">http://www.rubyist.net/~akr/chkbuild/debian/ruby-trunk/log/20120109T000400Z.log.html.gz</a></p>
Ruby master - Feature #5820 (Closed): Merge Onigmo to Ruby 2.0
https://redmine.ruby-lang.org/issues/5820
2011-12-29T02:42:18Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Ruby 1.9 では正規表現エンジンや M17N の基盤として Oniguruma を用いています。<br>
これを 2.0 では Oniguruma の改造版である、k-takata さんの Onigmo に置き換えようという話です。<br>
<a href="https://github.com/k-takata/Onigmo/tree/tmp/ruby-2.0.x" class="external">https://github.com/k-takata/Onigmo/tree/tmp/ruby-2.0.x</a></p>
<p>この取り込みによる影響は以下の通りです。</p>
<ul>
<li>100%互換 (既存のテストが全て無修正で通る)</li>
<li>いくつかの新機能 <a href="/issues/5208">[ruby-dev:44410]</a>
<ul>
<li>正規表現<br>
* \K, \R, \X, (?(cond)yes|no), \g<0>, \g<+n>, (?au)<br>
* Perl 5.10互換の名前参照(←Rubyには不要でしょう。)
<ul>
<li>Shift_JIS, EUC-JPで、全角アルファベットなどの大文字小文字同一視検索に対応。</li>
<li>Shift_JIS, EUC-JPで、\p{Han}, \p{Latin}, \p{Greek}, \p{Cyrillic} に対応。</li>
<li>最適化
<ul>
<li>暗黙のアンカーによる最適化を実装。</li>
<li>
<a href="http://redmine.ruby-lang.org/issues/3568" class="external">http://redmine.ruby-lang.org/issues/3568</a> で無効化された最適化を再度有効化。</li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
</ul>
<p>現状は POSIX 文字クラスに非互換があり、それが解決されればマージ可能と認識しています。</p>
Ruby master - Bug #5813 (Closed): net/http's EOFError and Keep-Alive
https://redmine.ruby-lang.org/issues/5813
2011-12-27T14:56:52Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p><a href="https://blade.ruby-lang.org/ruby-dev/39421">[ruby-dev:39421]</a> describes exceptions thrown by open-uri, and raise a question why net/http raises EOFError.</p>
<p>net/http sometimes raises EOFError.<br>
I recently find it is because of Keep-Alive.<br>
On HTTP/1.1, connections are Keep-Alive and a Keep-Alive connection has a timeout.<br>
If a client of such connection doesn't send anything after some communication,<br>
server closes the connection because of Keep-Alive timeout,<br>
and the client's connection shall raise EOFError (sometimes it may be ECONNRESET).</p>
<p>HTTP/1.1 says a client should retry a request if the request is idempotent.<br>
<a href="http://tools.ietf.org/html/rfc2616#section-8.1.4" class="external">http://tools.ietf.org/html/rfc2616#section-8.1.4</a><br>
<a href="http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-17#section-6.1.5" class="external">http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-17#section-6.1.5</a><br>
<a href="http://www.studyinghttp.net/connections" class="external">http://www.studyinghttp.net/connections</a><br>
So I attached a patch to such retry to net/http.</p>
<p>FYI, this timeout of Keep-Alive, KeepAliveTimeout, is:<br>
Apache in FreeBSD ports or pkgsrc is 5 seconds,<br>
the on in Debian Packages or RPM is 15 seconds.</p>
<p>diff --git a/lib/net/http.rb b/lib/net/http.rb<br>
index 879cfe0..13bd1a7 100644<br>
--- a/lib/net/http.rb<br>
+++ b/lib/net/http.rb<br>
@@ -1332,7 +1332,10 @@ module Net #:nodoc:<br>
res<br>
end</p>
<ul>
<li>IDEMPOTENT_METHODS_ = %w/GET HEAD PUT DELETE OPTIONS TRACE/ # :nodoc:</li>
<li>def transport_request(req)</li>
<li>
<pre><code> count = 0
begin_transport req
res = catch(:response) {
req.exec @socket, @curr_http_version, edit_path(req.path)
</code></pre>
</li>
</ul>
<p>@@ -1346,6 +1349,16 @@ module Net #:nodoc:<br>
}<br>
end_transport req, res<br>
res</p>
<ul>
<li>rescue EOFError, Errno::ECONNRESET => exception</li>
<li>
<pre><code> if count == 0 && IDEMPOTENT_METHODS_.include?(req.method)
</code></pre>
</li>
<li>
<pre><code> count += 1
</code></pre>
</li>
<li>
<pre><code> @socket.close if @socket and not @socket.closed?
</code></pre>
</li>
<li>
<pre><code> D "Conn close because of error #{exception}, and retry"
</code></pre>
</li>
<li>
<pre><code> retry
</code></pre>
</li>
<li>
<pre><code> end
</code></pre>
</li>
<li>
<pre><code> D "Conn close because of error #{exception}"
</code></pre>
</li>
<li>
<pre><code> @socket.close if @socket and not @socket.closed?
</code></pre>
</li>
<li>
<pre><code> raise
</code></pre>
rescue => exception<br>
D "Conn close because of error #{exception}"<br>
@socket.close if @socket and not @socket.closed?<br>
diff --git a/test/net/http/test_http.rb b/test/net/http/test_http.rb<br>
index 1515854..2e7ab4e 100644<br>
--- a/test/net/http/test_http.rb<br>
+++ b/test/net/http/test_http.rb<br>
@@ -564,3 +564,29 @@ class TestNetHTTPContinue < Test::Unit::TestCase<br>
assert_not_match(/HTTP/1.1 100 continue/, @debug.string)<br>
end<br>
end</li>
<li>
</ul>
<p>+class TestNetHTTPKeepAlive < Test::Unit::TestCase</p>
<ul>
<li>CONFIG = {</li>
<li>'host' => '127.0.0.1',</li>
<li>'port' => 10081,</li>
<li>'proxy_host' => nil,</li>
<li>'proxy_port' => nil,</li>
<li>'RequestTimeout' => 0.1,</li>
<li>}</li>
<li>
<li>include TestNetHTTPUtils</li>
<li>
<li>def test_keep_alive_get</li>
<li>start {|http|</li>
<li>
<pre><code> res = http.get('/')
</code></pre>
</li>
<li>
<pre><code> assert_kind_of Net::HTTPResponse, res
</code></pre>
</li>
<li>
<pre><code> assert_kind_of String, res.body
</code></pre>
</li>
<li>
<pre><code> sleep 1
</code></pre>
</li>
<li>
<pre><code> assert_nothing_raised {
</code></pre>
</li>
<li>
<pre><code> res = http.get('/')
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> assert_kind_of Net::HTTPResponse, res
</code></pre>
</li>
<li>
<pre><code> assert_kind_of String, res.body
</code></pre>
</li>
<li>}</li>
<li>end<br>
+end<br>
diff --git a/test/net/http/utils.rb b/test/net/http/utils.rb<br>
index 50f616f..07e0b9f 100644<br>
--- a/test/net/http/utils.rb<br>
+++ b/test/net/http/utils.rb<br>
@@ -51,6 +51,7 @@ module TestNetHTTPUtils<br>
:ServerType => Thread,<br>
}<br>
server_config[:OutputBufferSize] = 4 if config('chunked')</li>
<li>server_config[:RequestTimeout] = config('RequestTimeout') if config('RequestTimeout')<br>
if defined?(OpenSSL) and config('ssl_enable')<br>
server_config.update({<br>
:SSLEnable => true,</li>
</ul>
Ruby master - Bug #5790 (Closed): net/http の EOFError と Keep-Alive
https://redmine.ruby-lang.org/issues/5790
2011-12-22T18:49:13Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p><a href="https://blade.ruby-lang.org/ruby-dev/39421">[ruby-dev:39421]</a> がずっと心に残っていたので、思い立って調べてみたので、<br>
(正確には自分が高頻度で踏むようになったので調べてみた)<br>
その調査結果と対策案を提案します。</p>
<p>まず、投げられる原因ですが、根本的な原因は Keep-Alive のタイムアウトです。<br>
HTTP/1.1 ではデフォルトで持続的接続を行うので、複数回のリクエストに渡って<br>
一つの socket が使い回されます。</p>
<p>しかし、リクエスト同士で時間が開いていると、サーバー側でタイムアウトする<br>
可能性があります。この時にクライアント側の read(2) が 0 を返す、<br>
つまり EOFError となることがあります。</p>
<p>HTTP/1.1 は、冪等なメソッドの場合には確認なしにリトライすべきと言っているので、<br>
そのようにするパッチを添付します。<br>
冪等でないメソッドの場合にどうするべきかは悩ましいところです。<br>
<a href="http://tools.ietf.org/html/rfc2616#section-8.1.4" class="external">http://tools.ietf.org/html/rfc2616#section-8.1.4</a><br>
<a href="http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-17#section-6.1.5" class="external">http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-17#section-6.1.5</a><br>
<a href="http://www.studyinghttp.net/connections" class="external">http://www.studyinghttp.net/connections</a></p>
<p>なお、この Keep-Alive における Timeout は、<br>
Apache の場合、FreeBSD ports や pkgsrc では 5 秒、<br>
Debian Packages や RPM では 15 秒でした。</p>
<p>diff --git a/lib/net/http.rb b/lib/net/http.rb<br>
index 879cfe0..13bd1a7 100644<br>
--- a/lib/net/http.rb<br>
+++ b/lib/net/http.rb<br>
@@ -1332,7 +1332,10 @@ module Net #:nodoc:<br>
res<br>
end</p>
<ul>
<li>IDEMPOTENT_METHODS_ = %w/GET HEAD PUT DELETE OPTIONS TRACE/ # :nodoc:</li>
<li>def transport_request(req)</li>
<li>
<pre><code> count = 0
begin_transport req
res = catch(:response) {
req.exec @socket, @curr_http_version, edit_path(req.path)
</code></pre>
</li>
</ul>
<p>@@ -1346,6 +1349,16 @@ module Net #:nodoc:<br>
}<br>
end_transport req, res<br>
res</p>
<ul>
<li>rescue EOFError, Errno::ECONNRESET => exception</li>
<li>
<pre><code> if count == 0 && IDEMPOTENT_METHODS_.include?(req.method)
</code></pre>
</li>
<li>
<pre><code> count += 1
</code></pre>
</li>
<li>
<pre><code> @socket.close if @socket and not @socket.closed?
</code></pre>
</li>
<li>
<pre><code> D "Conn close because of error #{exception}, and retry"
</code></pre>
</li>
<li>
<pre><code> retry
</code></pre>
</li>
<li>
<pre><code> end
</code></pre>
</li>
<li>
<pre><code> D "Conn close because of error #{exception}"
</code></pre>
</li>
<li>
<pre><code> @socket.close if @socket and not @socket.closed?
</code></pre>
</li>
<li>
<pre><code> raise
</code></pre>
rescue => exception<br>
D "Conn close because of error #{exception}"<br>
@socket.close if @socket and not @socket.closed?<br>
diff --git a/test/net/http/test_http.rb b/test/net/http/test_http.rb<br>
index 1515854..2e7ab4e 100644<br>
--- a/test/net/http/test_http.rb<br>
+++ b/test/net/http/test_http.rb<br>
@@ -564,3 +564,29 @@ class TestNetHTTPContinue < Test::Unit::TestCase<br>
assert_not_match(/HTTP/1.1 100 continue/, @debug.string)<br>
end<br>
end</li>
<li>
</ul>
<p>+class TestNetHTTPKeepAlive < Test::Unit::TestCase</p>
<ul>
<li>CONFIG = {</li>
<li>'host' => '127.0.0.1',</li>
<li>'port' => 10081,</li>
<li>'proxy_host' => nil,</li>
<li>'proxy_port' => nil,</li>
<li>'RequestTimeout' => 0.1,</li>
<li>}</li>
<li>
<li>include TestNetHTTPUtils</li>
<li>
<li>def test_keep_alive_get</li>
<li>start {|http|</li>
<li>
<pre><code> res = http.get('/')
</code></pre>
</li>
<li>
<pre><code> assert_kind_of Net::HTTPResponse, res
</code></pre>
</li>
<li>
<pre><code> assert_kind_of String, res.body
</code></pre>
</li>
<li>
<pre><code> sleep 1
</code></pre>
</li>
<li>
<pre><code> assert_nothing_raised {
</code></pre>
</li>
<li>
<pre><code> res = http.get('/')
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> assert_kind_of Net::HTTPResponse, res
</code></pre>
</li>
<li>
<pre><code> assert_kind_of String, res.body
</code></pre>
</li>
<li>}</li>
<li>end<br>
+end<br>
diff --git a/test/net/http/utils.rb b/test/net/http/utils.rb<br>
index 50f616f..07e0b9f 100644<br>
--- a/test/net/http/utils.rb<br>
+++ b/test/net/http/utils.rb<br>
@@ -51,6 +51,7 @@ module TestNetHTTPUtils<br>
:ServerType => Thread,<br>
}<br>
server_config[:OutputBufferSize] = 4 if config('chunked')</li>
<li>server_config[:RequestTimeout] = config('RequestTimeout') if config('RequestTimeout')<br>
if defined?(OpenSSL) and config('ssl_enable')<br>
server_config.update({<br>
:SSLEnable => true,</li>
</ul>