Ruby Issue Tracking System: Issues
https://redmine.ruby-lang.org/
https://redmine.ruby-lang.org/favicon.ico?1711330511
2020-12-26T11:04:18Z
Ruby Issue Tracking System
Redmine
Ruby master - Feature #17472 (Rejected): HashWithIndifferentAccess like Hash extension
https://redmine.ruby-lang.org/issues/17472
2020-12-26T11:04:18Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Rails has <a href="https://api.rubyonrails.org/classes/ActiveSupport/HashWithIndifferentAccess.html" class="external">ActiveSupport::HashWithIndifferentAccess</a>, which is widely used in Rails to handle Request, Session, ActionView's form construction, ActiveRecord's DB communication, and so on. It receives String or Symbol and normalize them to fetch the value. But it is implemented with Ruby. If we provide C implementation of that, Rails will gain the performance improvement.</p>
<p>summary of previous discussion: <a href="https://github.com/rails/rails/pull/40182#issuecomment-687607812" class="external">https://github.com/rails/rails/pull/40182#issuecomment-687607812</a></p>
Ruby master - Feature #17468 (Closed): Deprecate RUBY_DEVEL
https://redmine.ruby-lang.org/issues/17468
2020-12-25T09:32:50Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Some configuration of Ruby use RUBY_DEVEL, which depends PATCH_LEVEL.<br>
But depending PATCH_LEVEL causes issues which will become revealed on the final release.<br>
Though we release some previews and RCs, they don't contributes the quality around RUBY_DEVEL.</p>
<p>Therefore to ensure CI tests the quality of the final release, we need to deprecate RUBY_DEVEL.</p>
Ruby master - Misc #17376 (Assigned): Reduce number of GitHub Actions
https://redmine.ruby-lang.org/issues/17376
2020-12-08T09:45:17Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>At this time we have 127 checks for GitHub commits, but unfortunately GitHub UI only shows 100 checks.<br>
It sometimes makes we don't see a failed check.<br>
Could you reduce number of GitHub actions at least less than 100?</p>
Ruby master - Feature #16275 (Closed): Revert `.:` syntax
https://redmine.ruby-lang.org/issues/16275
2019-10-23T15:26:13Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p><code>obj.:method</code> is introduced at r66667 by <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Proposal: Shorthand operator for Object#method (Open)" href="https://redmine.ruby-lang.org/issues/12125">#12125</a> and <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Syntax sugar for method reference (Closed)" href="https://redmine.ruby-lang.org/issues/13581">#13581</a>.<br>
It encourages the functional programming style in Ruby.</p>
<p>But this shorthand syntax is just for methods of <code>self</code> without arguments.<br>
It causes another feature requests like <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Proposal: Shorthand operator for "#instance_method" (Open)" href="https://redmine.ruby-lang.org/issues/16273">#16273</a> (and lambda compositions like <a class="issue tracker-1 status-7 priority-4 priority-default closed" title="Bug: Refactor Proc#>> and #<< (Feedback)" href="https://redmine.ruby-lang.org/issues/15428">#15428</a>).</p>
<p>Such features will introduce a new view of Ruby but no one illustrates the whole picture yet.<br>
I worried about such patch work may cause a conflict with future expansion of functional programing style or a just a garbage feature.</p>
<p><code>.:</code> syntax is introduced in 2.7.0 preview1, not released in production yet.<br>
How about reverting at this time and re-introduce with a big picture.</p>
Ruby master - Feature #16131 (Closed): Remove $SAFE, taint and trust
https://redmine.ruby-lang.org/issues/16131
2019-08-29T07:14:34Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Ruby had Taint checking which is originally introduced in Perl.<br>
<a href="https://en.wikipedia.org/wiki/Taint_checking" class="external">https://en.wikipedia.org/wiki/Taint_checking</a></p>
<p>It was intended to provide a useful tool for handle objects which are come from outside.<br>
Input data is set as tainted by default and call untaint if you checked or filtered the value.<br>
Some people used this feature in the age of CGI.</p>
<p>But these days, no one use the mechanism and input libraries usually doesn't support it.<br>
For example rack, as following shows its input is not tainted and the mechanism is unusable.</p>
<pre><code>% cat foo.ru
run ->(env) do
['200', {'Content-Type' => 'text/plain'}, ["Is QUERY_STRING tainted?: #{env["QUERY_STRING"].tainted?}"]]
end
% rackup foo.ru
[51724] Puma starting in cluster mode...
[51724] * Version 3.12.1 (ruby 2.6.3-p62), codename: Llamas in Pajamas
[51724] * Min threads: 3, max threads: 3
[51724] * Environment: development
[51724] * Process workers: 1
[51724] * Preloading application
[51724] * Listening on tcp://localhost:9292
[51724] Use Ctrl-C to stop
[51737] + Gemfile in context: /Users/naruse/work/td-cdp-api/Gemfile
[51724] - Worker 0 (pid: 51737) booted, phase: 0
</code></pre>
<pre><code>% curl http://localhost:9292/\?foo=1
Is QUERY_STRING tainted?: false
</code></pre>
<p>Therefore I think Taint checking mechanism is unusable on the current Ruby ecosystem.</p>
<p>On the other hand we experienced multiple vulnerability around $SAFE and taint mechanism.<br>
<a href="https://cse.google.com/cse?q=taint&cx=008288045305770251182%3Afvruzsaknew&ie=UTF-8" class="external">https://cse.google.com/cse?q=taint&cx=008288045305770251182%3Afvruzsaknew&ie=UTF-8</a><br>
The cost of maintaining it is expensive.</p>
<p>In conclusion, I think the taint mechanism is too expensive to maintain for the merit of it.<br>
I suggest to remove it.</p>
Ruby master - Feature #15958 (Closed): Time#inspect with frac
https://redmine.ruby-lang.org/issues/15958
2019-06-25T13:12:18Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>When Matz changed Time#to_s format in 2006, frac part of Time string is dropped because it is considered not useful.<br>
<a href="https://blade.ruby-lang.org/ruby-dev/29440">[ruby-dev:29440]</a></p>
<p>But recently we encounters some troubles the comparison of Time objects whose frac parts are different.<br>
<a href="https://twitter.com/__gfx__/status/1143056072422219776" class="external">https://twitter.com/__gfx__/status/1143056072422219776</a></p>
<p>For example a is original object and b is once stored in RDB (and dropped the frac part).<br>
Or there're multiple time objects which are generated in a single HTTP request.<br>
Of course they are different and assert_equal will be failed but inspect doesn't show the frac part.</p>
Ruby master - Misc #14861 (Closed): DevelopersMeeting20180718Japan
https://redmine.ruby-lang.org/issues/14861
2018-06-21T05:58:04Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Date: 2018/07/18 (Thu)<br>
Time: 14:00-18:00 (JST)<br>
Place: MoneyForward HQ (beware: new location) (Tokyo, Japan)<br>
Sign-up: <a href="https://ruby.connpass.com/event/92314/" class="external">https://ruby.connpass.com/event/92314/</a><br>
log: <a href="https://docs.google.com/document/d/1_cKh0LJd18y5CH1MfM6WC1fqh2rpHHZntrBCLTbwRSE/edit" class="external">https://docs.google.com/document/d/1_cKh0LJd18y5CH1MfM6WC1fqh2rpHHZntrBCLTbwRSE/edit</a><br>
pub: <a href="https://docs.google.com/document/d/1_cKh0LJd18y5CH1MfM6WC1fqh2rpHHZntrBCLTbwRSE/pub" class="external">https://docs.google.com/document/d/1_cKh0LJd18y5CH1MfM6WC1fqh2rpHHZntrBCLTbwRSE/pub</a></p>
<p>Please comment your favorite ticket numbers you want to ask to discuss with your <em>SHORT</em> comment or summary.<br>
(your summary/comment will help us because we don't need to read all of ticket comments)</p>
<p><em>DO NOT</em> discuss then on this ticket, please.</p>
<p>Past meetings: <a href="https://bugs.ruby-lang.org/projects/ruby/wiki#Developer-Meetings" class="external">https://bugs.ruby-lang.org/projects/ruby/wiki#Developer-Meetings</a></p>
<a name="NOTE"></a>
<h1 >NOTE<a href="#NOTE" class="wiki-anchor">¶</a></h1>
<p>Dev meeting <em>IS NOT</em> a decision making place. All decisions should be done at the bug tracker.<br>
Dev meeting is a place we can ask Matz, nobu, nurse and other developers directly.<br>
Matz is a very busy person. Take this opportunity to ask him. If you can not attend, other attendees can ask instead of you (if attendees can understand your issue).<br>
We will write a log about discussion to a file or to each ticket in English.<br>
All activities are best-effort (keep in mind that most of us are volunteer developers).<br>
The date, time and place is scheduled according to when/where we can reserve Matz's time.</p>
<a name="Agenda"></a>
<h1 >Agenda<a href="#Agenda" class="wiki-anchor">¶</a></h1>
<a name="Next-dev-meeting"></a>
<h2 >Next dev-meeting<a href="#Next-dev-meeting" class="wiki-anchor">¶</a></h2>
<a name="About-26-timeframe"></a>
<h2 >About 2.6 timeframe<a href="#About-26-timeframe" class="wiki-anchor">¶</a></h2>
<a name="Carry-over-from-previous-meetings"></a>
<h2 >Carry-over from previous meeting(s)<a href="#Carry-over-from-previous-meetings" class="wiki-anchor">¶</a></h2>
<ul>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Comparable#clamp with a range (Closed)" href="https://redmine.ruby-lang.org/issues/14784">#14784</a>] One-sided Comparable#clamp (with endless/startless ranges) (zverok)
<ul>
<li>more reasonable version of Object#enumerate proposed for the previous meeting.</li>
</ul>
</li>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: [PATCH] implement Timeout in VM (Open)" href="https://redmine.ruby-lang.org/issues/14859">#14859</a>] Timeout in VM (normalperson)
<ul>
<li>Still needs some work, mainly wondering if the idea of moving this part of stdlib into core VM is acceptable or not. No semantic changes except speed improvement.</li>
</ul>
</li>
</ul>
<a name="From-Attendees"></a>
<h2 >From Attendees<a href="#From-Attendees" class="wiki-anchor">¶</a></h2>
<p>(will be edited later)<br>
(if you have a write access permission, please list directly)</p>
<ul>
<li>[Bug <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Array#delete_if does not use #delete (Closed)" href="https://redmine.ruby-lang.org/issues/14887">#14887</a>] Array#delete_if does not use #delete (shyouhei)
<ul>
<li>Is it by design, or a bug?</li>
</ul>
</li>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Readline: expose rl_completion_quote_character variable (Closed)" href="https://redmine.ruby-lang.org/issues/13050">#13050</a>] Readline: expose rl_completion_quote_character variable (nobu)</li>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Add official API for setting timezone on Time (Closed)" href="https://redmine.ruby-lang.org/issues/14850">#14850</a>] Add official API for setting timezone on Time (nobu)</li>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Proposal to add Hash#=== (Open)" href="https://redmine.ruby-lang.org/issues/14869">#14869</a>] Proposal to add Hash#=== (nobu)</li>
<li>[Feature <a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Calculate age in Date class (Rejected)" href="https://redmine.ruby-lang.org/issues/14877">#14877</a>] Calculate age in Date class (nobu)</li>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Add Range#subrange? (Closed)" href="https://redmine.ruby-lang.org/issues/14473">#14473</a>] Add Range#subrange? (tarui)</li>
<li>[Bug <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: Enumerator::Lazy creates unnecessary Array objects. (Closed)" href="https://redmine.ruby-lang.org/issues/14908">#14908</a>] Enumerator::Lazy creates unnecessary Array objects. (nobu)</li>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Introduce pattern matching syntax (Closed)" href="https://redmine.ruby-lang.org/issues/14912">#14912</a>] Introduce pattern matching syntax (mame)
<ul>
<li>pattern matching</li>
</ul>
</li>
<li>opt_to_s (nobu)
<ul>
<li>see below for his patch</li>
</ul>
</li>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Add String#byteslice! (Open)" href="https://redmine.ruby-lang.org/issues/13626">#13626</a>] Add String#byteslice! (aycabta)</li>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Add display width method to String for CLI (Open)" href="https://redmine.ruby-lang.org/issues/14618">#14618</a>] Add display width method to String for CLI (aycabta)</li>
<li>[Misc <a class="issue tracker-5 status-2 priority-4 priority-default" title="Misc: Add RDoc documents to tar ball (Assigned)" href="https://redmine.ruby-lang.org/issues/14917">#14917</a>] Add RDoc documents to tar ball (aycabta)</li>
<li>[Feature <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: Use Reline for fallback of ext/readline (Closed)" href="https://redmine.ruby-lang.org/issues/14918">#14918</a>] Use Reline for fallback of ext/readline (aycabta)</li>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Add String#byteinsert (Open)" href="https://redmine.ruby-lang.org/issues/14919">#14919</a>] Add String#byteinsert (aycabta)</li>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Proposal to add Array#=== (Open)" href="https://redmine.ruby-lang.org/issues/14916">#14916</a>] Proposal to add Array#=== (aycabta)</li>
</ul>
<a name="From-non-attendees"></a>
<h2 >From non-attendees<a href="#From-non-attendees" class="wiki-anchor">¶</a></h2>
<p>(will be edited later)<br>
(if you have a write access, please list directly)</p>
<ul>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: ArgumentErrorが発生した時メソッドのプロトタイプをメッセージに含む (Open)" href="https://redmine.ruby-lang.org/issues/14111">#14111</a>] ArgumentErrorが発生した時メソッドのプロトタイプをメッセージに含む (esjee)
<ul>
<li>Suggestion: include method parameters when generating an ArgumentsError message. Nobu's patch in the issue provides functionality to make writing a ruby gem to provide this functionality possible.</li>
</ul>
</li>
<li>[Bug <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Add command line argument to deactivate JIT (Closed)" href="https://redmine.ruby-lang.org/issues/14878">#14878</a>] Add command line argument to deactivate JIT (k0kubun)
<ul>
<li>Please discuss the necessity of the flag and its name in the proposal.</li>
</ul>
</li>
<li>[Feature <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Implement String #blank? #present? and improve #strip and family to handle unicode (Open)" href="https://redmine.ruby-lang.org/issues/12306">#12306</a>] Implement String #blank? #present? and improve #strip and family to handle unicode (sam.saffron)</li>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Extend case to match several values at once (Closed)" href="https://redmine.ruby-lang.org/issues/14913">#14913</a>] Extend case to match several values at once (zverok)
<ul>
<li>some steps towards better pattern matching</li>
</ul>
</li>
<li>[Feature <a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Add BasicObject#instance_exec_with_block (Rejected)" href="https://redmine.ruby-lang.org/issues/14914">#14914</a>] Add BasicObject#instance_exec_with_block (jeremyevans0)</li>
<li>[Feature <a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Deprecate String#crypt (Rejected)" href="https://redmine.ruby-lang.org/issues/14915">#14915</a>] Deprecate String#crypt, move implementation to string/crypt (jeremyevans0)</li>
</ul>
Ruby master - Misc #14770 (Open): [META] DevelopersMeeting
https://redmine.ruby-lang.org/issues/14770
2018-05-17T12:28:50Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>A meta ticket to organize DevelopersMeeting tickets.<br>
<a href="https://bugs.ruby-lang.org/projects/ruby/wiki#Developer-Meetings" class="external">https://bugs.ruby-lang.org/projects/ruby/wiki#Developer-Meetings</a></p>
Ruby master - Feature #13881 (Open): Use getcontext/setcontext on OS X
https://redmine.ruby-lang.org/issues/13881
2017-09-08T09:10:50Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>getcontext/setcontext is first appeared on OS X 10.5 but deprecated on 10.6.<br>
It seems because POSIX removed them from recent specs.</p>
<p>IEEE Std 1003.1, 2004 Edition says makecontext's use of function declarators with empty parentheses<br>
is an obsolescent feature.<br>
<a href="http://pubs.opengroup.org/onlinepubs/009695399/functions/makecontext.html" class="external">http://pubs.opengroup.org/onlinepubs/009695399/functions/makecontext.html</a></p>
<p>Then POSIX.1-2008 removed those functions.</p>
<p>But OS X 10.13 still has them maybe because some essential applications uses them for co-routines.<br>
Therefore we can use them for performance.</p>
<pre><code>diff --git a/configure.in b/configure.in
index 08e109317f..3e75eb3cf2 100644
--- a/configure.in
+++ b/configure.in
@@ -1142,8 +1142,6 @@ AS_CASE(["$target_os"],
ac_cv_header_syscall_h=no
])
AS_IF([test $macosx_10_5 = yes], [
- ac_cv_func_getcontext=no
- ac_cv_func_setcontext=no
], [
AC_DEFINE(BROKEN_SETREUID, 1)
AC_DEFINE(BROKEN_SETREGID, 1)
diff --git a/cont.c b/cont.c
index c86095775c..f94883ef02 100644
--- a/cont.c
+++ b/cont.c
@@ -65,7 +65,15 @@
#ifndef _WIN32
#include <unistd.h>
#include <sys/mman.h>
-#include <ucontext.h>
+# ifdef __APPLE__
+/* avoid deprecated maks on ucontext.h */
+int getcontext(ucontext_t *);
+void makecontext(ucontext_t *, void (*)(), int, ...);
+int setcontext(const ucontext_t *);
+int swapcontext(ucontext_t * __restrict, const ucontext_t * __restrict);
+# else
+# include <ucontext.h>
+# endif
#endif
#define RB_PAGE_SIZE (pagesize)
#define RB_PAGE_MASK (~(RB_PAGE_SIZE - 1))
</code></pre>
Ruby master - Feature #13873 (Closed): Optimize Dir.glob with FNM_EXTGLOB
https://redmine.ruby-lang.org/issues/13873
2017-09-05T17:51:27Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Rails often queries Dir.glob with complex pattern.</p>
<p>On accessing Rails site, it calls glob as follows on querying templates.<br>
Note that Dir[] insists File::FNM_EXTGLOB.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="k">def</span> <span class="nf">find_template_paths</span><span class="p">(</span><span class="n">query</span><span class="p">)</span>
<span class="no">Dir</span><span class="p">[</span><span class="n">query</span><span class="p">].</span><span class="nf">uniq</span><span class="p">.</span><span class="nf">reject</span> <span class="k">do</span> <span class="o">|</span><span class="n">filename</span><span class="o">|</span>
<span class="no">File</span><span class="p">.</span><span class="nf">directory?</span><span class="p">(</span><span class="n">filename</span><span class="p">)</span> <span class="o">||</span>
<span class="c1"># deals with case-insensitive file systems.</span>
<span class="o">!</span><span class="no">File</span><span class="p">.</span><span class="nf">fnmatch</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="n">filename</span><span class="p">,</span> <span class="no">File</span><span class="o">::</span><span class="no">FNM_EXTGLOB</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
<p><a href="https://github.com/rails/rails/blob/6f1c18308ebffc97d51440cdeed7be71de58f26a/actionview/lib/action_view/template/resolver.rb#L247" class="external">https://github.com/rails/rails/blob/6f1c18308ebffc97d51440cdeed7be71de58f26a/actionview/lib/action_view/template/resolver.rb#L247</a></p>
<p>For example on accessing index page of Post Model (/posts), it calls following glob:<br>
Dir["/home/naruse/foo/app/views/posts/index{.en,}{.html,.text,.js,.css,.ics,.csv,.vcf,.png,.jpeg,.gif,.bmp,.tiff,.svg,.mpeg,.xml,.rss,.atom,.yaml,.multipart_form,.url_encoded_form,.json,.pdf,.zip,.gzip,}{}{.raw,.erb,.html,.builder,.ruby,.coffee,.jbuilder,}"]'</p>
<p>It calls many stat as follows:</p>
<pre><code>% sudo strace ./miniruby58882 -e'Dir["/home/naruse/foo/app/views/posts/index{.en,}{.html,.text,.js,.css,.ics,.csv,.vcf,.png,.jpeg,.gif,.bmp,.tiff,.svg,.mpeg,.xml,.rss,.atom,.yaml,.multipart_form,.url_encoded_form,.json,.pdf,.zip,.gzip,}{}{.raw,.erb,.html,.builder,.ruby,.coffee,.jbuilder,}"]'|&grep newfstatat|wc -l
400
</code></pre>
<p>Because current glob implementation handles braces in early stage,<br>
then calls stat for each combination of brace patterns instead of using readdir results.</p>
<p>Below is a patch to use readdir instead of hundreds of syscalls.<br>
Note that this patch changes the order of the result as follows:</p>
<pre><code> 1) Failure:
TestDir#test_glob [/home/naruse/ruby/test/ruby/test_dir.rb:157]:
<["/tmp/test-all/__test_dir__20170906-8620-ka8yqu/}}{}",
"/tmp/test-all/__test_dir__20170906-8620-ka8yqu/}}a"]> expected but was
<["/tmp/test-all/__test_dir__20170906-8620-ka8yqu/}}a",
"/tmp/test-all/__test_dir__20170906-8620-ka8yqu/}}{}"]>.
</code></pre>
<pre><code>diff --git a/dir.c b/dir.c
index b7afaec4e0..7ca84fa8c5 100644
--- a/dir.c
+++ b/dir.c
@@ -290,6 +290,8 @@ bracket(
#define UNESCAPE(p) (escape && *(p) == '\\' ? (p) + 1 : (p))
#define ISEND(p) (!*(p) || (pathname && *(p) == '/'))
#define RETURN(val) return *pcur = p, *scur = s, (val);
+#define FNMATCH_ALLOC_N(type, n) ((type *)malloc(sizeof(type) * (n)))
+#define FNMATCH_FREE(ptr) free(ptr)
static int
fnmatch_helper(
@@ -348,6 +350,56 @@ fnmatch_helper(
}
goto failed;
}
+
+ case '{': {
+ size_t len = pend - p;
+ char *buf = FNMATCH_ALLOC_N(char, len);
+ const char *rbrace = NULL;
+ while (p < pend) {
+ const char *t = ++p;
+ int nest = 0;
+ while (p < pend && !(*p == ',' && nest == 0)) {
+ if (*p == '{') nest++;
+ if (*p == '}') {
+ if (nest == 0) {
+ if (!rbrace) rbrace = p;
+ goto rest;
+ }
+ nest--;
+ }
+ if (*p == '\\' && escape) {
+ if (++p >= pend) break;
+ }
+ Inc(p, pend, enc);
+ }
+ if (!rbrace) {
+ rbrace = p;
+ while (rbrace < pend && !(*rbrace == '}' && nest == 0)) {
+ if (*rbrace == '{') nest++;
+ if (*rbrace == '}') nest--;
+ if (*rbrace == '\\' && escape) {
+ if (++p >= pend) break;
+ }
+ Inc(rbrace, pend, enc);
+ }
+ }
+rest:
+ memcpy(buf, t, p-t);
+ buf[p-t]=0;
+ strlcpy(buf+(p-t), rbrace+1, len-(p-t));
+ {
+ const char *pp = buf, *ss = s;
+ r = fnmatch_helper((const char **)&pp, &ss, flags|FNM_DOTMATCH, enc);
+ }
+ if (r == 0) {
+ p = buf;
+ FNMATCH_FREE(buf);
+ RETURN(0);
+ }
+ if (p >= rbrace) break;
+ }
+ FNMATCH_FREE(buf);
+ }
}
/* ordinary */
@@ -1426,6 +1478,12 @@ has_magic(const char *p, const char *pend, int flags, rb_encoding *enc)
case '[':
return MAGICAL;
+ case '{':
+ if (flags & FNM_EXTGLOB) {
+ return MAGICAL;
+ }
+ break;
+
case '\\':
if (escape && p++ >= pend)
continue;
@@ -2272,6 +2330,13 @@ push_pattern(const char *path, VALUE ary, void *enc)
rb_ary_push(ary, name);
}
+struct push_glob_args {
+ struct glob_args glob;
+ int flags;
+ int fd;
+};
+static int push_caller(const char *path, VALUE val, void *enc);
+
static int
ruby_brace_expand(const char *str, int flags, ruby_glob_func *func, VALUE arg,
rb_encoding *enc, VALUE var)
@@ -2280,7 +2345,7 @@ ruby_brace_expand(const char *str, int flags, ruby_glob_func *func, VALUE arg,
const char *p = str;
const char *pend = p + strlen(p);
const char *s = p;
- const char *lbrace = 0, *rbrace = 0;
+ const char *lbrace = NULL, *rbrace = NULL;
int nest = 0, status = 0;
while (*p) {
@@ -2299,9 +2364,18 @@ ruby_brace_expand(const char *str, int flags, ruby_glob_func *func, VALUE arg,
if (lbrace && rbrace) {
size_t len = strlen(s) + 1;
- char *buf = GLOB_ALLOC_N(char, len);
+ char *buf;
long shift;
+ if (func == push_caller && !strchr(lbrace, '/')) {
+ /* Now it reaches file basename entry. */
+ /* Handle braces in glob_helper */
+ struct push_glob_args *a = (struct push_glob_args *)arg;
+ a->flags |= FNM_EXTGLOB;
+ return glob_call_func(func, s, arg, enc);
+ }
+
+ buf = GLOB_ALLOC_N(char, len);
if (!buf) return -1;
memcpy(buf, s, lbrace-s);
shift = (lbrace-s);
@@ -2365,12 +2439,6 @@ ruby_brace_glob(const char *str, int flags, ruby_glob_func *func, VALUE arg)
return ruby_brace_glob_with_enc(str, flags, func, arg, rb_ascii8bit_encoding());
}
-struct push_glob_args {
- struct glob_args glob;
- int flags;
- int fd;
-};
-
static int
push_caller(const char *path, VALUE val, void *enc)
{
</code></pre>
Ruby master - Feature #13869 (Open): Filter non directories from Dir.glob
https://redmine.ruby-lang.org/issues/13869
2017-09-05T14:30:58Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Dir.glob is a tool to fetch filesystem entries with filtering.</p>
<p>On Rails, it often query files from template directories with braces (FNM_EXTGLOB)<br>
<a href="https://github.com/rails/rails/blob/6f1c18308ebffc97d51440cdeed7be71de58f26a/actionview/lib/action_view/template/resolver.rb#L247" class="external">https://github.com/rails/rails/blob/6f1c18308ebffc97d51440cdeed7be71de58f26a/actionview/lib/action_view/template/resolver.rb#L247</a></p>
<pre><code class="ruby syntaxhl" data-language="ruby"> <span class="k">def</span> <span class="nf">find_template_paths</span><span class="p">(</span><span class="n">query</span><span class="p">)</span>
<span class="no">Dir</span><span class="p">[</span><span class="n">query</span><span class="p">].</span><span class="nf">uniq</span><span class="p">.</span><span class="nf">reject</span> <span class="k">do</span> <span class="o">|</span><span class="n">filename</span><span class="o">|</span>
<span class="no">File</span><span class="p">.</span><span class="nf">directory?</span><span class="p">(</span><span class="n">filename</span><span class="p">)</span> <span class="o">||</span>
<span class="c1"># deals with case-insensitive file systems.</span>
<span class="o">!</span><span class="no">File</span><span class="p">.</span><span class="nf">fnmatch</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="n">filename</span><span class="p">,</span> <span class="no">File</span><span class="o">::</span><span class="no">FNM_EXTGLOB</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
<p>By this code, File.directory?() will call lstat(2) system call for each files.</p>
<p>But if Dir.glob is extended, it can avoid calling lstat because it can fetch entry's type from struct dirent<br>
(on many platforms).</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/dir.c b/dir.c
index b7afaec4e0..5db9c2dc79 100644
</span><span class="gd">--- a/dir.c
</span><span class="gi">+++ b/dir.c
</span><span class="p">@@ -206,6 +206,7 @@</span> typedef enum {
#else
#define FNM_SHORTNAME 0
#endif
<span class="gi">+#define FNM_NONDIR 0x40
</span>
#define FNM_NOMATCH 1
#define FNM_ERROR 2
<span class="p">@@ -1408,7 +1409,7 @@</span> do_opendir(const int basefd, const char *path, int flags, rb_encoding *enc,
}
/* Globing pattern */
<span class="gd">-enum glob_pattern_type { PLAIN, ALPHA, MAGICAL, RECURSIVE, MATCH_ALL, MATCH_DIR };
</span><span class="gi">+enum glob_pattern_type { PLAIN, ALPHA, MAGICAL, RECURSIVE, MATCH_ALL, MATCH_DIR, MATCH_NONDIR };
</span>
/* Return nonzero if S has any special globbing chars in it. */
static enum glob_pattern_type
<span class="p">@@ -1581,7 +1582,7 @@</span> glob_make_pattern(const char *p, const char *e, int flags, rb_encoding *enc)
glob_free_pattern(list);
return 0;
}
<span class="gd">- tmp->type = dirsep ? MATCH_DIR : MATCH_ALL;
</span><span class="gi">+ tmp->type = dirsep ? MATCH_DIR : flags & flags & FNM_NONDIR ? MATCH_NONDIR : MATCH_ALL;
</span> tmp->str = 0;
*tail = tmp;
tmp->next = 0;
<span class="p">@@ -1893,7 +1894,7 @@</span> glob_helper(
struct stat st;
int status = 0;
struct glob_pattern **cur, **new_beg, **new_end;
<span class="gd">- int plain = 0, magical = 0, recursive = 0, match_all = 0, match_dir = 0;
</span><span class="gi">+ int plain = 0, magical = 0, recursive = 0, match_all = 0, match_dir = 0, match_nondir = 0;
</span> int escape = !(flags & FNM_NOESCAPE);
size_t pathlen = baselen + namelen;
const char *base = path;
<span class="p">@@ -1926,6 +1927,9 @@</span> glob_helper(
case MATCH_DIR:
match_dir = 1;
break;
<span class="gi">+ case MATCH_NONDIR:
+ match_nondir = 1;
+ break;
</span> case RECURSIVE:
rb_bug("continuous RECURSIVEs");
}
<span class="p">@@ -1940,7 +1944,7 @@</span> glob_helper(
pathtype = path_noent;
}
}
<span class="gd">- if (match_dir && (pathtype == path_unknown || pathtype == path_symlink)) {
</span><span class="gi">+ if ((match_dir || match_nondir) && (pathtype == path_unknown || pathtype == path_symlink)) {
</span> if (do_stat(fd, base, &st, flags, enc) == 0) {
pathtype = IFTODT(st.st_mode);
}
<span class="p">@@ -1953,6 +1957,11 @@</span> glob_helper(
status = glob_call_func(funcs->match, subpath, arg, enc);
if (status) return status;
}
<span class="gi">+ if (match_nondir && pathtype > path_noent && pathtype != path_directory) {
+ const char *subpath = path + baselen + (baselen && path[baselen] == '/');
+ status = glob_call_func(funcs->match, subpath, arg, enc);
+ if (status) return status;
+ }
</span> if (match_dir && pathtype == path_directory) {
const char *subpath = path + baselen + (baselen && path[baselen] == '/');
char *tmp = join_path(subpath, namelen, dirsep, "", 0);
<span class="p">@@ -3152,4 +3161,10 @@</span> Init_Dir(void)
* on Microsoft Windows.
*/
rb_file_const("FNM_SHORTNAME", INT2FIX(FNM_SHORTNAME));
<span class="gi">+
+ /* Document-const: File::Constants::FNM_NONDIR
+ *
+ * Makes patterns to match non directory. Valid only
+ */
+ rb_file_const("FNM_NONDIR", INT2FIX(FNM_NONDIR));
</span> }
</code></pre>
Ruby master - Feature #13712 (Closed): String#start_with? with regexp
https://redmine.ruby-lang.org/issues/13712
2017-07-04T09:18:51Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>String#start_with? should receive regexp.</p>
<p>When I write a parser, I want to check a string is start with a pattern or not.<br>
It's just the same thing with <a href="https://ruby-doc.org/stdlib-2.4.0/libdoc/strscan/rdoc/StringScanner.html#method-i-match-3F" class="external">StringScanner#match</a></p>
<p>If I want to do the same thing with normal string method, it needs to write like <code>/\A#{re}/.match(…)</code>.<br>
But if re is argument, it needs to create a new temporary regexp every time.</p>
<p>Though we have a workaround as follows but it's bit tricky.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="s2">"foo "</span><span class="p">.</span><span class="nf">rindex</span><span class="p">(</span><span class="sr">/fo+./</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
</code></pre>
<p>A patch is following:</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/re.c b/re.c
index d0aa2a792e..f672ba75ec 100644
</span><span class="gd">--- a/re.c
</span><span class="gi">+++ b/re.c
</span><span class="p">@@ -1588,6 +1588,84 @@</span> rb_reg_search(VALUE re, VALUE str, long pos, int reverse)
return rb_reg_search0(re, str, pos, reverse, 1);
}
<span class="gi">+bool
+rb_reg_start_with_p(VALUE re, VALUE str)
+{
+ long pos = 0;
+ long result;
+ VALUE match;
+ struct re_registers regi, *regs = &regi;
+ regex_t *reg;
+ int tmpreg;
+ onig_errmsg_buffer err = "";
+
+ reg = rb_reg_prepare_re0(re, str, err);
+ tmpreg = reg != RREGEXP_PTR(re);
+ if (!tmpreg) RREGEXP(re)->usecnt++;
+
+ match = rb_backref_get();
+ if (!NIL_P(match)) {
+ if (FL_TEST(match, MATCH_BUSY)) {
+ match = Qnil;
+ }
+ else {
+ regs = RMATCH_REGS(match);
+ }
+ }
+ if (NIL_P(match)) {
+ MEMZERO(regs, struct re_registers, 1);
+ }
+ result = onig_match(reg,
+ (UChar*)(RSTRING_PTR(str)),
+ ((UChar*)(RSTRING_PTR(str)) + RSTRING_LEN(str)),
+ (UChar*)(RSTRING_PTR(str)),
+ regs, ONIG_OPTION_NONE);
+ if (!tmpreg) RREGEXP(re)->usecnt--;
+ if (tmpreg) {
+ if (RREGEXP(re)->usecnt) {
+ onig_free(reg);
+ }
+ else {
+ onig_free(RREGEXP_PTR(re));
+ RREGEXP_PTR(re) = reg;
+ }
+ }
+ if (result < 0) {
+ if (regs == &regi)
+ onig_region_free(regs, 0);
+ if (result == ONIG_MISMATCH) {
+ rb_backref_set(Qnil);
+ return false;
+ }
+ else {
+ onig_error_code_to_str((UChar*)err, (int)result);
+ rb_reg_raise(RREGEXP_SRC_PTR(re), RREGEXP_SRC_LEN(re), err, re);
+ }
+ }
+
+ if (NIL_P(match)) {
+ int err;
+ match = match_alloc(rb_cMatch);
+ err = rb_reg_region_copy(RMATCH_REGS(match), regs);
+ onig_region_free(regs, 0);
+ if (err) rb_memerror();
+ }
+ else {
+ FL_UNSET(match, FL_TAINT);
+ }
+
+ RMATCH(match)->str = rb_str_new4(str);
+ OBJ_INFECT(match, str);
+
+ RMATCH(match)->regexp = re;
+ RMATCH(match)->rmatch->char_offset_updated = 0;
+ rb_backref_set(match);
+
+ OBJ_INFECT(match, re);
+
+ return true;
+}
+
</span> VALUE
rb_reg_nth_defined(int nth, VALUE match)
{
<span class="gh">diff --git a/string.c b/string.c
index 072f1329ee..6542a4acb1 100644
</span><span class="gd">--- a/string.c
</span><span class="gi">+++ b/string.c
</span><span class="p">@@ -9126,6 +9126,7 @@</span> rb_str_rpartition(VALUE str, VALUE sep)
RSTRING_LEN(str)-pos-RSTRING_LEN(sep)));
}
<span class="gi">+extern bool rb_reg_start_with_p(VALUE re, VALUE str);
</span> /*
* call-seq:
* str.start_with?([prefixes]+) -> true or false
<span class="p">@@ -9146,11 +9147,20 @@</span> rb_str_start_with(int argc, VALUE *argv, VALUE str)
for (i=0; i<argc; i++) {
VALUE tmp = argv[i];
<span class="gd">- StringValue(tmp);
- rb_enc_check(str, tmp);
- if (RSTRING_LEN(str) < RSTRING_LEN(tmp)) continue;
- if (memcmp(RSTRING_PTR(str), RSTRING_PTR(tmp), RSTRING_LEN(tmp)) == 0)
- return Qtrue;
</span><span class="gi">+ switch (BUILTIN_TYPE(tmp)) {
+ case T_REGEXP:
+ {
+ bool r = rb_reg_start_with_p(tmp, str);
+ if (r) return Qtrue;
+ }
+ break;
+ default:
+ StringValue(tmp);
+ rb_enc_check(str, tmp);
+ if (RSTRING_LEN(str) < RSTRING_LEN(tmp)) continue;
+ if (memcmp(RSTRING_PTR(str), RSTRING_PTR(tmp), RSTRING_LEN(tmp)) == 0)
+ return Qtrue;
+ }
</span> }
return Qfalse;
}
</code></pre>
Ruby master - Feature #13608 (Rejected): Add TracePoint#thread
https://redmine.ruby-lang.org/issues/13608
2017-05-29T05:21:37Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>rb_trace_arg_t, TracePoint's internal struct, already stores the thread which the event happened at,<br>
but there's not API to fetch it.<br>
How about adding an API to get the info.</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/test/ruby/test_settracefunc.rb b/test/ruby/test_settracefunc.rb
index 53ee82a229..65dbc938ab 100644
</span><span class="gd">--- a/test/ruby/test_settracefunc.rb
</span><span class="gi">+++ b/test/ruby/test_settracefunc.rb
</span><span class="p">@@ -699,6 +699,23 @@</span> def test_tracepoint_enabled
assert_equal(false, trace.enabled?)
end
<span class="gi">+ def test_tracepoint_thread
+ trace = TracePoint.new(:call, :return){|tp|
+ next if !target_thread?
+ next if tp.path != __FILE__
+ assert_equal(Thread.current, tp.thread)
+ case tp.event
+ when :call
+ assert_raise(RuntimeError) {tp.return_value}
+ when :return
+ assert_equal("xyzzy", tp.return_value)
+ end
+ }
+ trace.enable{
+ foo
+ }
+ end
+
</span> def method_test_tracepoint_return_value obj
obj
end
<span class="gh">diff --git a/vm_trace.c b/vm_trace.c
index decb2c32e4..702b84e6e3 100644
</span><span class="gd">--- a/vm_trace.c
</span><span class="gi">+++ b/vm_trace.c
</span><span class="p">@@ -776,6 +776,12 @@</span> rb_tracearg_path(rb_trace_arg_t *trace_arg)
return trace_arg->path;
}
<span class="gi">+VALUE
+rb_tracearg_thread(rb_trace_arg_t *trace_arg)
+{
+ return trace_arg->th->self;
+}
+
</span> static void
fill_id_and_klass(rb_trace_arg_t *trace_arg)
{
<span class="p">@@ -913,6 +919,15 @@</span> tracepoint_attr_path(VALUE tpval)
}
/*
<span class="gi">+ * Thread of the event
+ */
+static VALUE
+tracepoint_attr_thread(VALUE tpval)
+{
+ return rb_tracearg_thread(get_trace_arg());
+}
+
+/*
</span> * Return the name at the definition of the method being called
*/
static VALUE
<span class="p">@@ -1502,6 +1517,7 @@</span> Init_vm_trace(void)
rb_define_method(rb_cTracePoint, "self", tracepoint_attr_self, 0);
rb_define_method(rb_cTracePoint, "return_value", tracepoint_attr_return_value, 0);
rb_define_method(rb_cTracePoint, "raised_exception", tracepoint_attr_raised_exception, 0);
<span class="gi">+ rb_define_method(rb_cTracePoint, "thread", tracepoint_attr_thread, 0);
</span>
rb_define_singleton_method(rb_cTracePoint, "stat", tracepoint_stat_s, 0);
</code></pre>
Ruby master - Feature #13577 (Assigned): Digest.file accidentally receives File object but uses f...
https://redmine.ruby-lang.org/issues/13577
2017-05-19T09:03:19Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Digest::SHA256.file()'s first argument is file path name but it accidentally accepts file object.<br>
But for file objects created with O_TMPFILE to_path returns the directory of the temporary file and this File.open will fail.</p>
<pre><code> class ::Digest::Class
# Creates a digest object and reads a given file, _name_.
# Optional arguments are passed to the constructor of the digest
# class.
#
# p Digest::SHA256.file("X11R6.8.2-src.tar.bz2").hexdigest
# # => "f02e3c85572dc9ad7cb77c2a638e3be24cc1b5bea9fdbb0b0299c9668475c534"
def self.file(name, *args)
new(*args).file(name)
end
end
module Instance
# Updates the digest with the contents of a given file _name_ and
# returns self.
def file(name)
File.open(name, "rb") {|f|
buf = ""
while f.read(16384, buf)
update buf
end
}
self
end
</code></pre>
Ruby master - Feature #13303 (Feedback): String#any? as !String#empty?
https://redmine.ruby-lang.org/issues/13303
2017-03-12T18:29:26Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Once I proposed "some container#nonempty?" on <a class="issue tracker-2 status-7 priority-4 priority-default closed" title="Feature: some container#nonempty? (Feedback)" href="https://redmine.ruby-lang.org/issues/12075">#12075</a>, and understand there's Array#any?.</p>
<p>Today I found String doesn't have such method.</p>
Ruby master - Misc #13027 (Closed): Release Engineering 2.4
https://redmine.ruby-lang.org/issues/13027
2016-12-12T14:43:37Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>META ticket for Release Engineering 2.4.</p>
<p>see also <a class="version" href="https://redmine.ruby-lang.org/versions/33">2.4</a></p>
<ul>
<li>current status: feature freeze</li>
<li>Q: Can I add a new feature?
<ul>
<li>A: No. Its status is already "feature freeze".</li>
</ul>
</li>
<li>Q: Can I fix a bug?
<ul>
<li>A: Yes</li>
</ul>
</li>
<li>Q: I want to complain about 2.4.0-rc1
<ul>
<li>A: create a ticket. It may change 2.4.0 RTM.</li>
</ul>
</li>
<li>Q: When ruby_2_4 branch is created?
<ul>
<li>A: scheduled on 2016-12-23</li>
</ul>
</li>
<li>Q: When Ruby 2.4.0 is released?
<ul>
<li>A: scheduled at 2016-12-25T00:00:00/03:00:00</li>
</ul>
</li>
</ul>
Ruby master - Feature #13020 (Closed): Zlib.gzip and Zlib.gunzip
https://redmine.ruby-lang.org/issues/13020
2016-12-09T22:37:22Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>I added Zlib.deflate/inflate [Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Add Zlib.deflate / Zlib.inflate (Closed)" href="https://redmine.ruby-lang.org/issues/4180">#4180</a>] before, but writing/reading gzip is still too complex.<br>
It should have shorthand method.</p>
Ruby master - Feature #12896 (Closed): Add compiler version message into rbconfig
https://redmine.ruby-lang.org/issues/12896
2016-11-04T06:30:53Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>When a extension library developer receives bug reports, they sometimes want to know what compiler is used for building the ruby binary.<br>
We can already know its compiler option but cannot know compiler name and version.</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/configure.in b/configure.in
index 3734afa..6793d7d 100644
</span><span class="gd">--- a/configure.in
</span><span class="gi">+++ b/configure.in
</span><span class="p">@@ -536,8 +536,10 @@</span> for option in --version -v -V -qversion; do
AS_CASE($cc_version_status, [0], [:], [continue])
AS_CASE($cc_version_message, [*Warning*], [continue])
cc_version='$(CC) '$option
<span class="gi">+ break
</span> done
AC_SUBST(CC_VERSION, $cc_version)
<span class="gi">+AC_SUBST(CC_VERSION_MESSAGE, $cc_version_message)
</span><span class="err">
</span> RUBY_UNIVERSAL_ARCH
if test "$target_cpu" != "$host_cpu" -a "$GCC" = yes -a "$cross_compiling" = no -a "$universal_binary" = no; then
</code></pre>
Ruby master - Misc #12751 (Open): Incompatibility of Ruby 3
https://redmine.ruby-lang.org/issues/12751
2016-09-12T06:38:38Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>META ticket for Ruby 3's breakages</p>
<ul>
<li>Encoding on Windows
<ul>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: On Windows use UTF-8 as filesystem encoding (Closed)" href="https://redmine.ruby-lang.org/issues/12654">#12654</a>]</li>
<li>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Use UTF-8 encoding for ENV on Windows (Closed)" href="https://redmine.ruby-lang.org/issues/12650">#12650</a>]</li>
</ul>
</li>
</ul>
Ruby master - Feature #12553 (Closed): IO.readlines(filename, chomp: true)
https://redmine.ruby-lang.org/issues/12553
2016-07-05T09:11:11Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>IO.readlinesやIO.foreach、IO#each_lineなどは戻り値(またはブロックパラメータ)のそれぞれの「行」に改行を含みます。</p>
<p>POSIXの "Line" の定義は末尾の改行を含んでのものなので非常に正しい挙動なのですが、正直不便です。</p>
<blockquote>
<p><a href="http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_206" class="external">3.206 Line</a></p>
<p>A sequence of zero or more non- characters plus a terminating character.</p>
</blockquote>
<p>例えば IO.readlines(filename, chomp: true) などで改行を最初から削ってくれませんか</p>
Ruby master - Misc #12474 (Third Party's Issue): Wishlist for Windows Unix compatibility features
https://redmine.ruby-lang.org/issues/12474
2016-06-09T08:37:29Z
naruse (Yui NARUSE)
naruse@airemix.jp
<a name="fork2"></a>
<h2 >fork(2)<a href="#fork2" class="wiki-anchor">¶</a></h2>
<p>Windows already has related features like <code>CreateProcess</code>, but for servers and workers fork is required for source code compatibility of existing scripts,<br>
which are written for Unix.</p>
<a name="nonblocking-IO"></a>
<h2 >nonblocking IO<a href="#nonblocking-IO" class="wiki-anchor">¶</a></h2>
<ul>
<li>To check socket's availability by <code>IO#read_nonblock</code> <a href="https://github.com/ruby/ruby/pull/1089" class="external">https://github.com/ruby/ruby/pull/1089</a>
</li>
<li><code>O_NONBLOCK</code></li>
</ul>
<a name="Signal"></a>
<h2 >Signal<a href="#Signal" class="wiki-anchor">¶</a></h2>
<p>Windows doesn't support true Signal.<br>
<a href="https://msdn.microsoft.com/en-us//library/xdkz3x12.aspx" class="external">https://msdn.microsoft.com/en-us//library/xdkz3x12.aspx</a></p>
<a name="tz"></a>
<h2 >tz<a href="#tz" class="wiki-anchor">¶</a></h2>
<p>Get IANA timezone name of current timezone from OS.</p>
<a name="Remove-a-file-which-is-opened-by-another-process"></a>
<h2 >Remove a file which is opened by another process<a href="#Remove-a-file-which-is-opened-by-another-process" class="wiki-anchor">¶</a></h2>
<p>If the process opens the file with <code>CreateFile</code> with <code>FILE_SHARE_DELETE</code> flag, another process can remove the file.<br>
But in that case processes can't remove its parent folder.<br>
<a href="https://bugs.ruby-lang.org/issues/11218" class="external">https://bugs.ruby-lang.org/issues/11218</a></p>
<a name="symlink"></a>
<h2 >symlink<a href="#symlink" class="wiki-anchor">¶</a></h2>
<p>Windows has mklink and SeCreateSymbolicLinkPrivilege, but it has some limitations.<br>
<a href="http://k-takata.o.oo7.jp/diary/2013-04.html#03" class="external">http://k-takata.o.oo7.jp/diary/2013-04.html#03</a><br>
<a href="https://twitter.com/n0kada/status/570232516545638400" class="external">https://twitter.com/n0kada/status/570232516545638400</a></p>
<a name="Bug-18882-On-64-mingw-ucrt-Fileread-sometimes-doesnt-read-entire-file-alanwu"></a>
<h2 >[Bug <a class="issue tracker-1 status-6 priority-4 priority-default closed" title="Bug: File.read cuts off a text file with special characters when reading it on MS Windows (Rejected)" href="https://redmine.ruby-lang.org/issues/18882">#18882</a>] On 64-mingw-ucrt, File.read() sometimes doesn't read entire file (alanwu)<a href="#Bug-18882-On-64-mingw-ucrt-Fileread-sometimes-doesnt-read-entire-file-alanwu" class="wiki-anchor">¶</a></h2>
<p><a href="https://github.com/ruby/dev-meeting-log/blob/master/DevMeeting-2022-07-21.md#bug-18882-on-64-mingw-ucrt-fileread-sometimes-doesnt-read-entire-file-alanwu" class="external">https://github.com/ruby/dev-meeting-log/blob/master/DevMeeting-2022-07-21.md#bug-18882-on-64-mingw-ucrt-fileread-sometimes-doesnt-read-entire-file-alanwu</a></p>
<a name="fd"></a>
<h2 >fd<a href="#fd" class="wiki-anchor">¶</a></h2>
<a name="Generic-handling-of-file-pipe-socket-by-fd"></a>
<h3 >Generic handling of file, pipe, socket by fd<a href="#Generic-handling-of-file-pipe-socket-by-fd" class="wiki-anchor">¶</a></h3>
<a name="fd-passing-to-child-process-other-than-012"></a>
<h3 >fd passing to child process other than 0,1,2<a href="#fd-passing-to-child-process-other-than-012" class="wiki-anchor">¶</a></h3>
<a name="Get-access-mode-from-fdHANDLE"></a>
<h3 >Get access mode from fd/HANDLE<a href="#Get-access-mode-from-fdHANDLE" class="wiki-anchor">¶</a></h3>
<a name="System-side-append-mode"></a>
<h3 >System side append mode<a href="#System-side-append-mode" class="wiki-anchor">¶</a></h3>
<p><a href="https://bugs.ruby-lang.org/issues/18605" class="external">https://bugs.ruby-lang.org/issues/18605</a></p>
<a name="a-high-performance-selector-API"></a>
<h3 >a high performance selector API<a href="#a-high-performance-selector-API" class="wiki-anchor">¶</a></h3>
<p><a href="https://tonyarcieri.com/a-gentle-introduction-to-nio4r" class="external">https://tonyarcieri.com/a-gentle-introduction-to-nio4r</a></p>
<a name="socket"></a>
<h2 >socket<a href="#socket" class="wiki-anchor">¶</a></h2>
<p><code>CloseHandle</code> should work with socket.</p>
<a name="writev2"></a>
<h2 >writev(2)<a href="#writev2" class="wiki-anchor">¶</a></h2>
<p><a href="https://twitter.com/okuoku/status/670212493134852097" class="external">https://twitter.com/okuoku/status/670212493134852097</a><br>
<a href="https://msdn.microsoft.com/ja-jp/library/windows/desktop/aa365749%28v=vs.85%29.aspx" class="external">https://msdn.microsoft.com/ja-jp/library/windows/desktop/aa365749%28v=vs.85%29.aspx</a></p>
<a name="ANSI-color-code"></a>
<h2 >ANSI color code<a href="#ANSI-color-code" class="wiki-anchor">¶</a></h2>
<p><a href="http://srad.jp/story/16/02/09/0639223/" class="external">http://srad.jp/story/16/02/09/0639223/</a></p>
<a name="UNIXSocket"></a>
<h2 ><del>UNIXSocket</del><a href="#UNIXSocket" class="wiki-anchor">¶</a></h2>
<p><del>We can emuate UNIXSocket with named pipe?</del><br>
Implemented [Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Support `UNIXSocket` on Windows (Closed)" href="https://redmine.ruby-lang.org/issues/19135">#19135</a>]</p>
Ruby master - Feature #12336 (Closed): Use -std=gnu99 instead of -std=c99 for GCC
https://redmine.ruby-lang.org/issues/12336
2016-05-01T09:58:53Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>r36038以降、Ruby本体をビルドする際にはGCCに-stdオプションを与えており、<br>
若干の変更を経て現在では-std=iso9899:1999 (-std=c99相当)を指定しています。</p>
<p>しかしこの指定の場合、GCCのC標準関数の代わりにビルトイン関数を用いる最適化が行われないため、<br>
現在ちょっと残念な状態が続いています。<br>
<a href="https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html" class="external">https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html</a></p>
<p>また、これの導入のきっかけはおそらくr36029の</p>
<ul>
<li>compile.c (iseq_set_sequence): nonstatic initializer of an aggregate type is a C99ism.</li>
<li>compile.c (enum compile_array_type_t): comma at the end of enum list is a C99ism.</li>
</ul>
<p>だと思うのですが、すでにc99相当なのであんまり効いてない……かな?<br>
もっともやりがちなMixed Declarationsは別に-Wdeclaration-after-statementがあるのでそれはそれで問題ありません。<br>
<a href="https://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html" class="external">https://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html</a><br>
<a href="https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html" class="external">https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html</a></p>
<p>現在だと、-std=gnu99でおおむね同レベルの警告を維持できると同時に、<br>
GCCのビルトイン関数を使えるようになってわずかな手間でちょっと早くなって嬉しいと思うのですがいかがでしょう。</p>
Ruby master - Misc #12283 (Closed): Obsolete ChangeLog and commit message in Git-style
https://redmine.ruby-lang.org/issues/12283
2016-04-14T18:57:41Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>How about Git-style commit message instead of ChangeLog?</p>
<a name="Proposal"></a>
<h2 >Proposal<a href="#Proposal" class="wiki-anchor">¶</a></h2>
<p>After we reach the consensus</p>
<ol>
<li>Write a commit message in Git-style.</li>
</ol>
<ul>
<li>Line 1: summary of commit (around 80 characters or less, <50 is better)</li>
<li>Line 2: empty line</li>
<li>Line 3-: More detailed explanation of commit</li>
</ul>
<p>See also: <a href="https://git-scm.com/book/ch5-2.html#Commit-Guidelines" class="external">https://git-scm.com/book/ch5-2.html#Commit-Guidelines</a></p>
<ol start="2">
<li>Stop manually edit ChangeLog and auto-generate on <code>make dist</code>.</li>
</ol>
<a name="Rationale"></a>
<h2 >Rationale<a href="#Rationale" class="wiki-anchor">¶</a></h2>
<p>A good commit message should include the objective of the commit.<br>
Committers should write "what purpose the change is for".<br>
In this sense, ChangeLog is not a good format. It lists per-file changes, so committers tend to write just "how the files were changed".</p>
<p>Also, ChangeLog causes commit conflicts frequently.<br>
Though it is not related with the actual commit.<br>
People who uses svn/git should see the log with it; ChangeLog is only for tarball users.</p>
<a name="Status"></a>
<h2 >Status<a href="#Status" class="wiki-anchor">¶</a></h2>
<p>mame made this proposal at the developers' meeting (13 Apr.), and there was no objection.</p>
<p>Matz agreed with this proposal, provided that a ChangeLog-like plain-text file is generated from the commit log.<br>
The release manager will create the file when Ruby is released.</p>
<a name="Note"></a>
<h3 >Note<a href="#Note" class="wiki-anchor">¶</a></h3>
<p>This ticket does NOT propose a migration from SVN to Git. Never discuss that in this ticket.</p>
Ruby master - Feature #12244 (Open): Add a way to `integer - integer % num`
https://redmine.ruby-lang.org/issues/12244
2016-04-02T13:45:33Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>We sometimes calculates <code>integer - integer % num</code>.</p>
<p>For example time series events into time partitions, we write code like</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">event</span> <span class="c1"># {time: 1459580435, name: "hoge", text: "Rawr!"}</span>
<span class="n">partition</span> <span class="o">=</span> <span class="n">event</span><span class="p">[</span><span class="ss">:time</span><span class="p">]</span> <span class="o">-</span> <span class="n">event</span><span class="p">[</span><span class="ss">:time</span><span class="p">]</span> <span class="o">%</span> <span class="n">num</span>
<span class="n">chunk</span> <span class="o">=</span> <span class="n">get_chunk</span><span class="p">(</span><span class="n">partition</span><span class="p">)</span>
<span class="n">chunk</span><span class="p">.</span><span class="nf">write</span> <span class="n">event</span>
</code></pre>
<p><a href="https://twitter.com/tagomoris/status/715814050534461440" class="external">https://twitter.com/tagomoris/status/715814050534461440</a><br>
<a href="https://twitter.com/tagomoris/status/715814961260457985" class="external">https://twitter.com/tagomoris/status/715814961260457985</a></p>
<p>The name is always issue.<br>
There are some suggestions likes Integer#adjust](<a href="https://twitter.com/cocoatomo/status/716088708655489024" class="external">https://twitter.com/cocoatomo/status/716088708655489024</a>).</p>
<p>kosaki says <a href="https://twitter.com/kosaki55tea/status/716059296186765312" class="external">Excel's FLOOR() function is FLOOR(number, significance)</a>.<br>
Therefore Ruby should be <a href="https://twitter.com/kosaki55tea/status/716106530114768897" class="external">Integer#floor(digits=1, significance: nil)</a>.<br>
<a href="https://twitter.com/kosaki55tea/status/716107516409581568" class="external">kosaki agrees this</a>.</p>
<p>I checked the speed of a half baked implementation..., but it's 10x slow...</p>
<pre><code>% time ./miniruby -e'i=10000000;while i>0;i-=1;1459497599.floor(significance: 3600);end'
./miniruby 6.58s user 0.02s system 99% cpu 6.596 total
#3 1459604131 22:35:31 naruse@windy:~/obj/ruby
% time ./miniruby -e'i=10000000;while i>0;i-=1;t=1459497599;t-t%3600;end'
./miniruby -e'i=10000000;while i>0;i-=1;t=1459497599;t-t%3600;end' 0.52s user 0.00s system 99% cpu 0.520 total
</code></pre>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/numeric.c b/numeric.c
index 37217a1..f6acfe3 100644
</span><span class="gd">--- a/numeric.c
</span><span class="gi">+++ b/numeric.c
</span><span class="p">@@ -4123,6 +4123,42 @@</span> int_dotimes(VALUE num)
<span class="err">
</span> /*
* call-seq:
<span class="gi">+ * int.floor([ndigits]) -> integer or float
+ *
+ * Rounds +int+ to a given precision in decimal digits (default 0 digits).
+ *
+ * Precision may be negative. Returns a floating point number when +ndigits+
+ * is positive, +self+ for zero, and round down for negative.
+ *
+ * 1.round #=> 1
+ * 1.round(2) #=> 1.0
+ * 15.round(-1) #=> 20
+ */
+
+static VALUE
+int_floor(int argc, VALUE* argv, VALUE num)
+{
+ static ID keyword_ids[1];
+ VALUE kwargs[1], ndigits, opt;
+ if (!keyword_ids[0]) {
+ CONST_ID(keyword_ids[0], "significance");
+ }
+
+ rb_scan_args(argc, argv, "01:", &ndigits, &opt);
+ if (!NIL_P(opt)) {
+ VALUE factor;
+ long a, b;
+ rb_get_kwargs(opt, keyword_ids, 0, 1, kwargs);
+ factor = kwargs[0];
+ a = FIX2LONG(num);
+ b = FIX2LONG(factor);
+ return LONG2FIX(a - a % b);
+ }
+ return Qnil;
+}
+
+/*
+ * call-seq:
</span> * int.round([ndigits]) -> integer or float
*
* Rounds +int+ to a given precision in decimal digits (default 0 digits).
<span class="p">@@ -4321,7 +4357,7 @@</span> Init_Numeric(void)
rb_define_method(rb_cInteger, "to_i", int_to_i, 0);
rb_define_method(rb_cInteger, "to_int", int_to_i, 0);
rb_define_method(rb_cInteger, "to_f", int_to_f, 0);
<span class="gd">- rb_define_method(rb_cInteger, "floor", int_to_i, 0);
</span><span class="gi">+ rb_define_method(rb_cInteger, "floor", int_floor, -1);
</span> rb_define_method(rb_cInteger, "ceil", int_to_i, 0);
rb_define_method(rb_cInteger, "truncate", int_to_i, 0);
rb_define_method(rb_cInteger, "round", int_round, -1);
</code></pre>
Ruby master - Feature #12225 (Rejected): Remove inline assemblers and always enables USE_MACHINE_...
https://redmine.ruby-lang.org/issues/12225
2016-03-28T08:20:39Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Current vm_exec.c stores pc an explicitly declared register to get PC.<br>
Since recent CPUs and compilers are very smart, we expect they optimizes their use of registers.</p>
<p>With following patch the benchmark becomes following:</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/vm_exec.c b/vm_exec.c
index 5e4ff94..6f7c1ad 100644
</span><span class="gd">--- a/vm_exec.c
</span><span class="gi">+++ b/vm_exec.c
</span><span class="p">@@ -15,23 +15,6 @@</span>
static void vm_analysis_insn(int insn);
#endif
<span class="gd">-#if VMDEBUG > 0
-#define DECL_SC_REG(type, r, reg) register type reg_##r
-
-#elif defined(__GNUC__) && defined(__x86_64__)
-#define DECL_SC_REG(type, r, reg) register type reg_##r __asm__("r" reg)
-
-#elif defined(__GNUC__) && defined(__i386__)
-#define DECL_SC_REG(type, r, reg) register type reg_##r __asm__("e" reg)
-
-#elif defined(__GNUC__) && defined(__powerpc64__)
-#define DECL_SC_REG(type, r, reg) register type reg_##r __asm__("r" reg)
-
-#else
-#define DECL_SC_REG(type, r, reg) register type reg_##r
-#endif
-/* #define DECL_SC_REG(r, reg) VALUE reg_##r */
-
</span> #if VM_DEBUG_STACKOVERFLOW
NORETURN(static void vm_stack_overflow_for_insn(void));
static void
<span class="p">@@ -49,41 +32,12 @@</span> vm_exec_core(rb_thread_t *th, VALUE initial)
{
#if OPT_STACK_CACHING
<span class="gd">-#if 0
-#elif __GNUC__ && __x86_64__ && !defined(__native_client__)
- DECL_SC_REG(VALUE, a, "12");
- DECL_SC_REG(VALUE, b, "13");
-#else
</span> register VALUE reg_a;
register VALUE reg_b;
#endif
<span class="gd">-#endif
</span>
<span class="gd">-#if defined(__GNUC__) && defined(__i386__)
- DECL_SC_REG(const VALUE *, pc, "di");
- DECL_SC_REG(rb_control_frame_t *, cfp, "si");
-#define USE_MACHINE_REGS 1
-
-#elif defined(__GNUC__) && defined(__x86_64__)
- DECL_SC_REG(const VALUE *, pc, "14");
-# if defined(__native_client__)
- DECL_SC_REG(rb_control_frame_t *, cfp, "13");
-# else
- DECL_SC_REG(rb_control_frame_t *, cfp, "15");
-# endif
-#define USE_MACHINE_REGS 1
-
-#elif defined(__GNUC__) && defined(__powerpc64__)
- DECL_SC_REG(const VALUE *, pc, "14");
- DECL_SC_REG(rb_control_frame_t *, cfp, "15");
-#define USE_MACHINE_REGS 1
-
-#else
</span> register rb_control_frame_t *reg_cfp;
const VALUE *reg_pc;
<span class="gd">-#endif
-
-#if USE_MACHINE_REGS
</span>
#undef RESTORE_REGS
#define RESTORE_REGS() \
<span class="p">@@ -98,7 +52,6 @@</span> vm_exec_core(rb_thread_t *th, VALUE initial)
#define GET_PC() (reg_pc)
#undef SET_PC
#define SET_PC(x) (reg_cfp->pc = REG_PC = (x))
<span class="gd">-#endif
</span>
#if OPT_TOKEN_THREADED_CODE || OPT_DIRECT_THREADED_CODE
#include "vmtc.inc"
</code></pre>
<pre><code>Speedup ratio: compare with the result of `ruby 2.4.0dev (2016-03-27 trunk 54303) [x86_64-linux]' (greater is better)
name built-ruby
loop_whileloop 1.016
vm1_attr_ivar* 0.991
vm1_attr_ivar_set* 0.976
vm1_block* 1.013
vm1_const* 0.924
vm1_ensure* 0.978
vm1_float_simple* 1.006
vm1_gc_short_lived* 1.011
vm1_gc_short_with_complex_long* 1.036
vm1_gc_short_with_long* 1.064
vm1_gc_short_with_symbol* 0.997
vm1_gc_wb_ary* 1.005
vm1_gc_wb_ary_promoted* 1.000
vm1_gc_wb_obj* 0.977
vm1_gc_wb_obj_promoted* 1.029
vm1_ivar* 1.054
vm1_ivar_set* 0.961
vm1_length* 1.019
vm1_lvar_init* 0.962
vm1_lvar_set* 0.991
vm1_neq* 0.976
vm1_not* 0.903
vm1_rescue* 0.983
vm1_simplereturn* 1.005
vm1_swap* 1.000
vm1_yield* 0.979
</code></pre>
<pre><code>additional example micro benchmark
BEFORE gcc 4.8:
Performance counter stats for './miniruby -e@v=42; n=100_000_000;while n>0; x=x|x; x=x|x;n-=1;end':
7218.124555 task-clock (msec) # 0.998 CPUs utilized
123 context-switches # 0.017 K/sec
2 cpu-migrations # 0.000 K/sec
906 page-faults # 0.126 K/sec
21374094581 cycles # 2.961 GHz
4469895839 stalled-cycles-frontend # 20.91% frontend cycles idle
<not supported> stalled-cycles-backend
55226298374 instructions # 2.58 insns per cycle
# 0.08 stalled cycles per insn
7805291103 branches # 1081.346 M/sec
200172514 branch-misses # 2.56% of all branches
7.230608341 seconds time elapsed
BEFORE gcc version 5.3.0 20151204 (Ubuntu 5.3.0-3ubuntu1~14.04):
Performance counter stats for './miniruby -e@v=42; n=100_000_000;while n>0; x=x|x; x=x|x;n-=1;end':
8054.736236 task-clock (msec) # 0.998 CPUs utilized
128 context-switches # 0.016 K/sec
2 cpu-migrations # 0.000 K/sec
895 page-faults # 0.111 K/sec
23776261112 cycles # 2.952 GHz
7078686240 stalled-cycles-frontend # 29.77% frontend cycles idle
<not supported> stalled-cycles-backend
53126508523 instructions # 2.23 insns per cycle
# 0.13 stalled cycles per insn
7505454893 branches # 931.806 M/sec
201181233 branch-misses # 2.68% of all branches
8.074872624 seconds time elapsed
AFTER gcc version 4.8.5 (Ubuntu 4.8.5-2ubuntu1~14.04.1):
Performance counter stats for './miniruby -e@v=42; n=100_000_000;while n>0; x=x|x; x=x|x;n-=1;end':
7267.867318 task-clock (msec) # 0.997 CPUs utilized
169 context-switches # 0.023 K/sec
1 cpu-migrations # 0.000 K/sec
899 page-faults # 0.124 K/sec
21563673390 cycles # 2.967 GHz
4952119471 stalled-cycles-frontend # 22.97% frontend cycles idle
<not supported> stalled-cycles-backend
53226715304 instructions # 2.47 insns per cycle
# 0.09 stalled cycles per insn
7805365852 branches # 1073.955 M/sec
200218594 branch-misses # 2.57% of all branches
7.286793973 seconds time elapsed
AFTER gcc version 5.3.0 20151204 (Ubuntu 5.3.0-3ubuntu1~14.04):
Performance counter stats for './miniruby -e@v=42; n=100_000_000;while n>0; x=x|x; x=x|x;n-=1;end':
7146.899779 task-clock (msec) # 0.998 CPUs utilized
166 context-switches # 0.023 K/sec
2 cpu-migrations # 0.000 K/sec
899 page-faults # 0.126 K/sec
21188099959 cycles # 2.965 GHz
4839187155 stalled-cycles-frontend # 22.84% frontend cycles idle
<not supported> stalled-cycles-backend
52525802838 instructions # 2.48 insns per cycle
# 0.09 stalled cycles per insn
7505329721 branches # 1050.152 M/sec
200175714 branch-misses # 2.67% of all branches
7.157645157 seconds time elapsed
</code></pre>
Ruby master - Feature #12208 (Closed): Improve ri command
https://redmine.ruby-lang.org/issues/12208
2016-03-23T05:22:46Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Ruby has well documented in rdoc.<br>
It also has a frontend CLI.</p>
<p>But people seems not using <code>ri</code>...</p>
<p>At first, write your request here to improve ri command!</p>
Ruby master - Feature #12075 (Feedback): some container#nonempty?
https://redmine.ruby-lang.org/issues/12075
2016-02-16T08:08:28Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>I sometimes write following code.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">ary</span> <span class="o">=</span> <span class="n">some_metho_returns_nil_or_empty_container</span><span class="p">()</span> <span class="c1"># nil or "" or [] or {}</span>
<span class="k">if</span> <span class="n">ary</span> <span class="o">&&</span> <span class="o">!</span><span class="n">ary</span><span class="p">.</span><span class="nf">empty?</span>
<span class="c1"># some code</span>
<span class="k">end</span>
</code></pre>
<p>But the condition <code>ary && !ary.empty?</code> is too long and complex.<br>
Though Ruby 2.3 introduces <code>&.</code>, but this can’t be written as <code>ary&.empty?</code>.</p>
<p>One idea is add <code>nonempty?</code> write as <code>ary&.nonempty?</code>.</p>
<p>akr: <code>nonempty?</code> is not good name because human is not good at handling</p>
<p>This discussion matches following core classes:</p>
<ul>
<li>String</li>
<li>Array</li>
<li>Hash</li>
</ul>
Ruby master - Feature #12010 (Closed): Exclude dot and dotdot from Dir#each
https://redmine.ruby-lang.org/issues/12010
2016-01-20T02:17:16Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p><code>Dir#each</code> and <code>Dir#read</code> (including <code>Dir.entries</code>, <code>Dir.foreach</code> and other methods) return <code>"."</code> and <code>".."</code> at first.<br>
But through the all real use case <code>"."</code> and <code>".."</code> are useless.<br>
How about excluding them?</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/dir.c b/dir.c
index 193b5be..4c23a2d 100644
</span><span class="gd">--- a/dir.c
</span><span class="gi">+++ b/dir.c
</span><span class="p">@@ -699,6 +699,8 @@</span> fundamental_encoding_p(rb_encoding *enc)
#else
# define READDIR(dir, enc) readdir((dir))
#endif
<span class="gi">+#define DIR_IS_DOT_OR_DOTDOT(dp) ((dp)->d_name[0] == '.' && \
+ ((dp)->d_name[1] == '\0' || ((dp)->d_name[1] == '.' && (dp)->d_name[2] == '\0')))
</span>
/*
* call-seq:
<span class="p">@@ -720,13 +722,12 @@</span> dir_read(VALUE dir)
GetDIR(dir, dirp);
errno = 0;
<span class="gd">- if ((dp = READDIR(dirp->dir, dirp->enc)) != NULL) {
- return rb_external_str_new_with_enc(dp->d_name, NAMLEN(dp), dirp->enc);
- }
- else {
- if (errno != 0) rb_sys_fail(0);
- return Qnil; /* end of stream */
</span><span class="gi">+ while ((dp = READDIR(dirp->dir, dirp->enc)) != NULL) {
+ if (!DIR_IS_DOT_OR_DOTDOT(dp))
+ return rb_external_str_new_with_enc(dp->d_name, NAMLEN(dp), dirp->enc);
</span> }
<span class="gi">+ if (errno != 0) rb_sys_fail(0);
+ return Qnil; /* end of stream */
</span> }
/*
<span class="p">@@ -764,6 +765,7 @@</span> dir_each(VALUE dir)
const char *name = dp->d_name;
size_t namlen = NAMLEN(dp);
VALUE path;
<span class="gi">+ if (DIR_IS_DOT_OR_DOTDOT(dp)) continue;
</span> #if NORMALIZE_UTF8PATH
if (norm_p && has_nonascii(name, namlen) &&
!NIL_P(path = rb_str_normalize_ospath(name, namlen))) {
<span class="gh">diff --git a/test/pathname/test_pathname.rb b/test/pathname/test_pathname.rb
index 2690a3f..33f0d44 100644
</span><span class="gd">--- a/test/pathname/test_pathname.rb
</span><span class="gi">+++ b/test/pathname/test_pathname.rb
</span><span class="p">@@ -1238,7 +1238,7 @@</span> def test_entries
with_tmpchdir('rubytest-pathname') {|dir|
open("a", "w") {}
open("b", "w") {}
<span class="gd">- assert_equal([Pathname("."), Pathname(".."), Pathname("a"), Pathname("b")], Pathname(".").entries.sort)
</span><span class="gi">+ assert_equal([Pathname("a"), Pathname("b")], Pathname(".").entries.sort)
</span> }
end
<span class="p">@@ -1248,7 +1248,7 @@</span> def test_each_entry
open("b", "w") {}
a = []
Pathname(".").each_entry {|v| a << v }
<span class="gd">- assert_equal([Pathname("."), Pathname(".."), Pathname("a"), Pathname("b")], a.sort)
</span><span class="gi">+ assert_equal([Pathname("a"), Pathname("b")], a.sort)
</span> }
end
<span class="p">@@ -1278,7 +1278,7 @@</span> def test_opendir
Pathname(".").opendir {|d|
d.each {|e| a << e }
}
<span class="gd">- assert_equal([".", "..", "a", "b"], a.sort)
</span><span class="gi">+ assert_equal(["a", "b"], a.sort)
</span> }
end
<span class="gh">diff --git a/test/ruby/test_dir.rb b/test/ruby/test_dir.rb
index 0cc5a6a..d3f6602 100644
</span><span class="gd">--- a/test/ruby/test_dir.rb
</span><span class="gi">+++ b/test/ruby/test_dir.rb
</span><span class="p">@@ -186,7 +186,7 @@</span> def test_glob_recursive
def assert_entries(entries)
entries.sort!
<span class="gd">- assert_equal(%w(. ..) + ("a".."z").to_a, entries)
</span><span class="gi">+ assert_equal(("a".."z").to_a, entries)
</span> end
def test_entries
</code></pre>
Ruby master - Feature #12005 (Closed): Unify Fixnum and Bignum into Integer
https://redmine.ruby-lang.org/issues/12005
2016-01-19T02:38:01Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>CRuby has two <code>Integer</code> classes, <code>Fixnum</code> and <code>Bignum</code>.<br>
But it is implementation detail.<br>
They should be seen as a single class <code>Integer</code> like <code>Flonum</code>.</p>
<a name="Compatibility-note"></a>
<h2 >Compatibility note<a href="#Compatibility-note" class="wiki-anchor">¶</a></h2>
<ul>
<li>Q: How do I check whether Fixnum and Bignum are unified or not?</li>
<li>A: check RUBY_INTEGER_UNIFICATION macro</li>
</ul>
Ruby master - Feature #11987 (Open): daemons can't show the backtrace of rb_bug
https://redmine.ruby-lang.org/issues/11987
2016-01-13T10:38:57Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Ruby shows backtrace and other information into stderr on rb_bug.<br>
But daemon process redirects stderr into /dev/null.<br>
How do I get the log?</p>
<p>NOTE: if I can reproduce this, I can use strace to get it. But issues are not always reproducible.</p>
Ruby master - Feature #11801 (Closed): rb_inspect shouldn't raise error even if calling inspect r...
https://redmine.ruby-lang.org/issues/11801
2015-12-10T04:22:06Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Current rb_inspect raises Encoding::Compatibility error if obj.inspect returns a string<br>
whose encoding is not compatible with resulted string's encoding.<br>
But the behavior is not useful on inspecting an array or a hash containing such strings<br>
because it prevents to dump them.</p>
<p>Following patch changes to escape such strings like String#inspect.</p>
<pre><code>diff --git a/object.c b/object.c
index d339ff9..1f73cb4 100644
--- a/object.c
+++ b/object.c
@@ -465,6 +465,7 @@ rb_any_to_s(VALUE obj)
return str;
}
+VALUE rb_str_escape(VALUE str);
/*
* If the default external encoding is ASCII compatible, the encoding of
* the inspected result must be compatible with it.
@@ -478,11 +479,11 @@ rb_inspect(VALUE obj)
rb_encoding *ext = rb_default_external_encoding();
if (!rb_enc_asciicompat(ext)) {
if (!rb_enc_str_asciionly_p(str))
- rb_raise(rb_eEncCompatError, "inspected result must be ASCII only if default external encoding is ASCII incompatible");
+ return rb_str_escape(str);
return str;
}
if (rb_enc_get(str) != ext && !rb_enc_str_asciionly_p(str))
- rb_raise(rb_eEncCompatError, "inspected result must be ASCII only or use the default external encoding");
+ return rb_str_escape(str);
return str;
}
diff --git a/string.c b/string.c
index e6df91d..319c516 100644
--- a/string.c
+++ b/string.c
@@ -5265,6 +5265,70 @@ rb_str_buf_cat_escaped_char(VALUE result, unsigned int c, int unicode_p)
return l;
}
+VALUE
+rb_str_escape(VALUE str)
+{
+ int encidx = ENCODING_GET(str);
+ rb_encoding *enc = rb_enc_from_index(encidx);
+ const char *p = RSTRING_PTR(str);
+ const char *pend = RSTRING_END(str);
+ const char *prev = p;
+ char buf[CHAR_ESC_LEN + 1];
+ VALUE result = rb_str_buf_new(0);
+ int unicode_p = rb_enc_unicode_p(enc);
+ int asciicompat = rb_enc_asciicompat(enc);
+
+ while (p < pend) {
+ unsigned int c, cc;
+ int n = rb_enc_precise_mbclen(p, pend, enc);
+ if (!MBCLEN_CHARFOUND_P(n)) {
+ if (p > prev) str_buf_cat(result, prev, p - prev);
+ n = rb_enc_mbminlen(enc);
+ if (pend < p + n)
+ n = (int)(pend - p);
+ while (n--) {
+ snprintf(buf, CHAR_ESC_LEN, "\\x%02X", *p & 0377);
+ str_buf_cat(result, buf, strlen(buf));
+ prev = ++p;
+ }
+ continue;
+ }
+ n = MBCLEN_CHARFOUND_LEN(n);
+ c = rb_enc_mbc_to_codepoint(p, pend, enc);
+ p += n;
+ switch (c) {
+ case '\n': cc = 'n'; break;
+ case '\r': cc = 'r'; break;
+ case '\t': cc = 't'; break;
+ case '\f': cc = 'f'; break;
+ case '\013': cc = 'v'; break;
+ case '\010': cc = 'b'; break;
+ case '\007': cc = 'a'; break;
+ case 033: cc = 'e'; break;
+ default: cc = 0; break;
+ }
+ if (cc) {
+ if (p - n > prev) str_buf_cat(result, prev, p - n - prev);
+ buf[0] = '\\';
+ buf[1] = (char)cc;
+ str_buf_cat(result, buf, 2);
+ prev = p;
+ }
+ else if (asciicompat && rb_enc_isascii(c, enc) && ISPRINT(c)) {
+ }
+ else {
+ if (p - n > prev) str_buf_cat(result, prev, p - n - prev);
+ rb_str_buf_cat_escaped_char(result, c, unicode_p);
+ prev = p;
+ }
+ }
+ if (p > prev) str_buf_cat(result, prev, p - prev);
+ ENCODING_CODERANGE_SET(result, rb_usascii_encindex(), ENC_CODERANGE_7BIT);
+
+ OBJ_INFECT_RAW(result, str);
+ return result;
+}
+
/*
* call-seq:
* str.inspect -> string
diff --git a/test/ruby/test_m17n.rb b/test/ruby/test_m17n.rb
index ca25f85..2e9e65b 100644
--- a/test/ruby/test_m17n.rb
+++ b/test/ruby/test_m17n.rb
@@ -278,7 +278,7 @@ def test_object_utf16_32_inspect
o = Object.new
[Encoding::UTF_16BE, Encoding::UTF_16LE, Encoding::UTF_32BE, Encoding::UTF_32LE].each do |e|
o.instance_eval "undef inspect;def inspect;'abc'.encode('#{e}');end"
- assert_raise(Encoding::CompatibilityError) { [o].inspect }
+ assert_equal '[abc]', [o].inspect
end
ensure
Encoding.default_internal = orig_int
@@ -302,13 +302,18 @@ def o.inspect
def o.inspect
"abc".encode(Encoding.default_external)
end
- assert_raise(Encoding::CompatibilityError) { [o].inspect }
+ assert_equal '[abc]', [o].inspect
Encoding.default_external = Encoding::US_ASCII
def o.inspect
"\u3042"
end
- assert_raise(Encoding::CompatibilityError) { [o].inspect }
+ assert_equal '[\u3042]', [o].inspect
+
+ def o.inspect
+ "\x82\xa0".force_encoding(Encoding::Windows_31J)
+ end
+ assert_equal '[\x{82A0}]', [o].inspect
ensure
Encoding.default_internal = orig_int
Encoding.default_external = orig_ext
diff --git a/core/array/shared/inspect.rb b/core/array/shared/inspect.rb
index dc30518..a169fd0 100644
--- a/core/array/shared/inspect.rb
+++ b/core/array/shared/inspect.rb
@@ -83,9 +83,9 @@ describe :array_inspect, shared: true do
it "raises if inspected result is not default external encoding" do
utf_16be = mock("utf_16be")
- utf_16be.should_receive(:inspect).and_return("utf_16be".encode!(Encoding::UTF_16BE))
+ utf_16be.should_receive(:inspect).and_return(%<"utf_16be \u3042">.encode!(Encoding::UTF_16BE))
- lambda { [utf_16be].send(@method) }.should raise_error(Encoding::CompatibilityError)
+ [utf_16be].send(@method).should == '["utf_16be \u3042"]'
end
end
end
</code></pre>
Ruby master - Feature #11577 (Open): Add encodeURIComponent compatible API for URI
https://redmine.ruby-lang.org/issues/11577
2015-10-09T13:40:10Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>How about adding encodeURIComponent/decodeURIComponent compatible API?</p>
<p>There's already have some methods:</p>
<ul>
<li>URI.escape: context aware but deprecated.</li>
<li>URIencode_www_form: application/x-www-form-urlencoded, which encodes spaces into '+'</li>
<li>URIencode_www_form_component: above component</li>
</ul>
<p>So it doesn't have non-form URI escape method.</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/lib/uri/common.rb b/lib/uri/common.rb
index 1444ae8..0017ae3 100644
</span><span class="gd">--- a/lib/uri/common.rb
</span><span class="gi">+++ b/lib/uri/common.rb
</span><span class="p">@@ -1,4 +1,5 @@</span>
<span class="gd">-#--
</span><span class="gi">+#
+# -*- frozen-string-literal: true -*-
</span> # = uri/common.rb
#
# Author:: Akira Yamada <akira@ruby-lang.org>
<span class="p">@@ -329,27 +330,72 @@</span> module URI
DEFAULT_PARSER.make_regexp(schemes)
end
<span class="gi">+ TBLENCURICOMP_ = {} # :nodoc:
</span> TBLENCWWWCOMP_ = {} # :nodoc:
<span class="gd">- 256.times do |i|
- TBLENCWWWCOMP_[i.chr] = '%%%02X' % i
- end
- TBLENCWWWCOMP_[' '] = '+'
- TBLENCWWWCOMP_.freeze
</span><span class="gi">+ TBLDECURICOMP_ = {} # :nodoc:
</span> TBLDECWWWCOMP_ = {} # :nodoc:
256.times do |i|
h, l = i>>4, i&15
<span class="gd">- TBLDECWWWCOMP_['%%%X%X' % [h, l]] = i.chr
- TBLDECWWWCOMP_['%%%x%X' % [h, l]] = i.chr
- TBLDECWWWCOMP_['%%%X%x' % [h, l]] = i.chr
- TBLDECWWWCOMP_['%%%x%x' % [h, l]] = i.chr
</span><span class="gi">+ c = i.chr.freeze
+ k = sprintf('%%%X%X', h, l).freeze
+ TBLENCURICOMP_[c] = TBLENCWWWCOMP_[c] = k
+ TBLDECURICOMP_[k] = TBLDECWWWCOMP_[k] = c
+ k = sprintf('%%%x%X', h, l).freeze
+ TBLDECURICOMP_[k] = TBLDECWWWCOMP_[k] = c
+ k = sprintf('%%%X%x', h, l).freeze
+ TBLDECURICOMP_[k] = TBLDECWWWCOMP_[k] = c
+ k = sprintf('%%%x%x', h, l).freeze
+ TBLDECURICOMP_[k] = TBLDECWWWCOMP_[k] = c
</span> end
<span class="gi">+ TBLENCWWWCOMP_[' '] = '+'
+ TBLENCWWWCOMP_.freeze
+ TBLENCURICOMP_.freeze
</span> TBLDECWWWCOMP_['+'] = ' '
TBLDECWWWCOMP_.freeze
<span class="gi">+ TBLDECURICOMP_.freeze
+
+ # Encode given +str+ to URL-encoded form data.
+ #
+ # This method doesn't convert *, -, ., 0-9, A-Z, _, a-z, but does convert SP
+ # (ASCII space) to + and converts others to %XX.
+ #
+ # If +enc+ is given, convert +str+ to the encoding before percent encoding.
+ #
+ # This is an implementation of
+ # http://www.ecma-international.org/ecma-262/6.0/#sec-encodeuricomponent-uricomponent
+ #
+ # See URI.encode_www_form_component, URI.encode_www_form
+ def self.encode_component(str)
+ str = str.to_s.dup
+ if !str.ascii_only?
+ enc = str.encoding
+ if enc != Encoding::ASCII_8BIT && enc != Encoding::UTF_8
+ str.encode!(Encoding::UTF_8, invalid: :replace, undef: :replace)
+ end
+ str.force_encoding(Encoding::ASCII_8BIT)
+ end
+ str.gsub!(/[^\-_.!~*'()0-9A-Za-z]/, TBLENCURICOMP_)
+ str.force_encoding(Encoding::US_ASCII)
+ end
+
+ # Decode given +str+ of URL-encoded form data.
+ #
+ #
+ # This doesn't decodes + to SP.
+ #
+ # This is an implementation of
+ # http://www.ecma-international.org/ecma-262/6.0/#sec-decodeuricomponent-encodeduricomponent
+ #
+ # See URI.decode_www_form_component, URI.decode_www_form
+ def self.decode_component(str, enc=Encoding::UTF_8)
+ raise ArgumentError, "invalid %-encoding (#{str})" if /%(?!\h\h)/ =~ str
+ str.b.gsub(/%\h\h/, TBLDECURICOMP_).force_encoding(enc)
+ end
</span>
HTML5ASCIIINCOMPAT = defined? Encoding::UTF_7 ? [Encoding::UTF_7, Encoding::UTF_16BE, Encoding::UTF_16LE,
Encoding::UTF_32BE, Encoding::UTF_32LE] : [] # :nodoc:
<span class="gd">- # Encode given +str+ to URL-encoded form data.
</span><span class="gi">+ # Encode given +str+ in application/x-www-form-urlencoded format.
</span> #
# This method doesn't convert *, -, ., 0-9, A-Z, _, a-z, but does convert SP
# (ASCII space) to + and converts others to %XX.
<span class="p">@@ -373,7 +419,7 @@</span> module URI
str.force_encoding(Encoding::US_ASCII)
end
<span class="gd">- # Decode given +str+ of URL-encoded form data.
</span><span class="gi">+ # Decode given +str+ in application/x-www-form-urlencoded format.
</span> #
# This decodes + to SP.
#
<span class="p">@@ -457,7 +503,7 @@</span> module URI
if isindex
if sep.empty?
val = key
<span class="gd">- key = ''
</span><span class="gi">+ key = String.new
</span> end
isindex = false
end
<span class="p">@@ -471,7 +517,7 @@</span> module URI
if val
val.gsub!(/\+|%\h\h/, TBLDECWWWCOMP_)
else
<span class="gd">- val = ''
</span><span class="gi">+ val = String.new
</span> end
ary << [key, val]
<span class="gh">diff --git a/test/uri/test_common.rb b/test/uri/test_common.rb
index 5620415..c6b5633 100644
</span><span class="gd">--- a/test/uri/test_common.rb
</span><span class="gi">+++ b/test/uri/test_common.rb
</span><span class="p">@@ -54,6 +54,34 @@</span> class TestCommon < Test::Unit::TestCase
assert_raise(NoMethodError) { Object.new.URI("http://www.ruby-lang.org/") }
end
<span class="gi">+ def test_encode_component
+ assert_equal("%00%20!%22%23%24%25%26'()*%2B%2C-.%2F09%3A%3B%3C%3D%3E%3F%40" \
+ "AZ%5B%5C%5D%5E_%60az%7B%7C%7D~",
+ URI.encode_component("\x00 !\"\#$%&'()*+,-./09:;<=>?@AZ[\\]^_`az{|}~"))
+ assert_equal("%E6%9F%8A", URI.encode_component(
+ "\x95\x41".force_encoding(Encoding::Shift_JIS)))
+ assert_equal("%E3%81%82", URI.encode_component(
+ "\x30\x42".force_encoding(Encoding::UTF_16BE)))
+ assert_equal("%E3%81%82", URI.encode_component(
+ "\e$B$\"\e(B".force_encoding(Encoding::ISO_2022_JP)))
+ end
+
+ def test_decode_component
+ assert_equal(" +!\"\#$%&'()*+,-./09:;<=>?@AZ[\\]^_`az{|}~",
+ URI.decode_component(
+ "%20+%21%22%23%24%25%26%27%28%29*%2B%2C-.%2F09%3A%3B%3C%3D%3E%3F%40" \
+ "AZ%5B%5C%5D%5E_%60az%7B%7C%7D%7E"))
+ assert_equal("\xA1\xA2".force_encoding(Encoding::EUC_JP),
+ URI.decode_component("%A1%A2", "EUC-JP"))
+ assert_equal("\xE3\x81\x82\xE3\x81\x82".force_encoding("UTF-8"),
+ URI.decode_component("\xE3\x81\x82%E3%81%82".force_encoding("UTF-8")))
+
+ assert_raise(ArgumentError){URI.decode_component("%")}
+ assert_raise(ArgumentError){URI.decode_component("%a")}
+ assert_raise(ArgumentError){URI.decode_component("x%a_")}
+ assert_nothing_raised(ArgumentError){URI.decode_component("x"*(1024*1024))}
+ end
+
</span> def test_encode_www_form_component
assert_equal("%00+%21%22%23%24%25%26%27%28%29*%2B%2C-.%2F09%3A%3B%3C%3D%3E%3F%40" \
"AZ%5B%5C%5D%5E_%60az%7B%7C%7D%7E",
</code></pre>
Ruby master - Feature #11558 (Closed): Time related C APIs
https://redmine.ruby-lang.org/issues/11558
2015-09-30T05:59:44Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Time関連のC APIを追加して欲しいです。<br>
具体的には以下のようなものが欲しいです。</p>
<p>struct timespecとoffsetを取って、Timeを返してください。<br>
VALUE rb_time_timespec_new(const struct timespec *ts, int offset);<br>
趣旨としては、[秒, ナノ秒, offset]からTimeを作って欲しいと言うことです。<br>
(rb_time_num_new(Rational, offset)とかだと手間かかる上に遅い)</p>
<p>現在時刻をstruct timespecで欲しいです。<br>
void timespec_now(struct timespec *ts);</p>
<p>既存の非公開APIを公開してください。<br>
struct tm * localtime_with_gmtoff_zone(const time_t *t, struct tm *result, long *gmtoff, const char **zone);<br>
time_t timegm_noleapsecond(struct tm *tm);<br>
void tm_add_offset(struct tm *tm, long diff);</p>
<p><a href="https://github.com/nurse/strptime/blob/v0.1.1/ext/strptime/ruby_time.c" class="external">https://github.com/nurse/strptime/blob/v0.1.1/ext/strptime/ruby_time.c</a></p>
Ruby master - Misc #11516 (Closed): Ruby 2.3.0 release engeneering
https://redmine.ruby-lang.org/issues/11516
2015-09-09T06:13:18Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>For the management of release blockers.</p>
<p>Current Phase:<br>
Next release: preview1</p>
<p>Q: When ruby_2_2 branch is created?<br>
A: On RC1</p>
Ruby master - Misc #11474 (Closed): [META] Call for Feature Proposals for Ruby 2.3
https://redmine.ruby-lang.org/issues/11474
2015-08-21T05:44:04Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>This is meta ticket for Call for Feature Proposals for Ruby 2.3.<br>
After you attach your slide to the related ticket, write ticket number as a comment here to avoid I miss it.</p>
Ruby master - Feature #11302 (Closed): Dir.entries and Dir.foreach without [".", ".."]
https://redmine.ruby-lang.org/issues/11302
2015-06-24T06:24:58Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Dir.entries returns an array of its content with "." and "..".<br>
But as far as I met, almost all cases don't need them.</p>
<p>How about adding such new method or options?</p>
Ruby master - Feature #11253 (Closed): rb_io_modestr_oflags for Ruby API
https://redmine.ruby-lang.org/issues/11253
2015-06-12T08:18:12Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>If you have a wrapper of IO.open, you may handle mode.<br>
For example on Windows, you may want to add 'b' / OBINARY to given mode.</p>
<p>But some modes has only integer form like O_CLOEXEC, O_EXCL.<br>
If so you need to convert from modestr to oflags</p>
<p>Therefore how about adding a Ruby API version of rb_io_modestr_oflags</p>
Ruby master - Feature #11251 (Closed): Thread#name and Thread#name=
https://redmine.ruby-lang.org/issues/11251
2015-06-12T02:48:43Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Threadの名前の取得・設定を行うAPIを追加しませんか。</p>
<p><a class="issue tracker-2 status-2 priority-4 priority-default" title="Feature: Thread.new without block. (Assigned)" href="https://redmine.ruby-lang.org/issues/6694">#6694</a> や <a class="issue tracker-2 status-2 priority-4 priority-default" title="Feature: Configuration for Thread/Fiber creation (Assigned)" href="https://redmine.ruby-lang.org/issues/6695">#6695</a> などで断続的に議論がなされ、r47670 で一部自動で名前が付けられるようになったりもしていますが、<br>
Rubyレベルでそれを見る手段は提供されていませんし、明示的に付けることも出来ません。</p>
<p>今でもニーズは結構あるようです。</p>
<blockquote>
<p>"Fluentdでどのスレッドがどのプラグインに属しているのかが簡単に分かるようになるんですよね”<br>
<a href="https://twitter.com/repeatedly/status/608855851131011073" class="external">https://twitter.com/repeatedly/status/608855851131011073</a><br>
"確かにRubyのスレッドに名前付けられたら良さそう。主にログで識別する用途で。"<br>
<a href="https://twitter.com/frsyuki/status/608863065598140417" class="external">https://twitter.com/frsyuki/status/608863065598140417</a></p>
</blockquote>
<pre><code>Linux (glibc 2.12+)
int pthread_setname_np(pthread_t thread, const char *name);
int pthread_getname_np(pthread_t thread, char *name, size_t len);
FreeBSD
void pthread_set_name_np(pthread_t, const char *);
NetBSD
int pthread_getname_np(pthread_t thread, char *name, size_t len);
int pthread_setname_np(pthread_t thread, const char *name, void *arg);
http://netbsd.gw.com/cgi-bin/man-cgi?pthread_setname_np++NetBSD-current
OS X
/usr/include/pthread/pthread.h:int pthread_getname_np(pthread_t,char*,size_t);
/usr/include/pthread/pthread.h:int pthread_setname_np(const char*);
</code></pre>
Ruby master - Feature #11220 (Rejected): strptime(%6N)
https://redmine.ruby-lang.org/issues/11220
2015-06-04T14:46:35Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>strftimeにはマイクロ秒などでの出力を指定する、%6N, %9Nというフォーマットがあります。<br>
一方で、パースを行うstrptimeにはそのような指定子が現在ありません。</p>
<p>そのようなものに対するニーズはぼちぼちあるようなので入れませんか。<br>
<a href="http://answers.splunk.com/answers/1946/time-format-and-subseconds.html" class="external">http://answers.splunk.com/answers/1946/time-format-and-subseconds.html</a><br>
<a href="https://twitter.com/nalsh/status/606414387352502272" class="external">https://twitter.com/nalsh/status/606414387352502272</a></p>
<pre><code>diff --git a/ext/date/date_strptime.c b/ext/date/date_strptime.c
index e318af1..89a6421 100644
--- a/ext/date/date_strptime.c
+++ b/ext/date/date_strptime.c
@@ -611,6 +611,33 @@ date__strptime_internal(const char *str, size_t slen,
recur("%a %b %e %H:%M:%S %Z %Y");
goto matched;
+ case '0': case '1': case '2': case '3': case '4':
+ case '5': case '6': case '7': case '8': case '9':
+ {
+ VALUE n;
+ size_t w;
+ int sign = 1;
+ size_t osi;
+ w = read_digits(&fmt[fi], &n, 1);
+ fi += w;
+ w = NUM2SIZET(n);
+
+ if (issign(str[si])) {
+ if (str[si] == '-')
+ sign = -1;
+ si++;
+ }
+ osi = si;
+ READ_DIGITS(n, w == 0 ? 9 : w)
+ if (sign == -1)
+ n = f_negate(n);
+ set_hash("sec_fraction",
+ rb_rational_new2(n,
+ f_expt(INT2FIX(10),
+ ULONG2NUM(si - osi))));
+ goto matched;
+ }
+
default:
if (str[si] != '%')
fail();
</code></pre>
Ruby master - Feature #11218 (Closed): File.open FILE_SHARE_DELETE
https://redmine.ruby-lang.org/issues/11218
2015-06-04T05:19:41Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>fluentdという、OSSのログコレクタがあるのですが、これには<code>in_tail</code>プラグインというものがあります。<br>
これは、ログファイルを監視して、ログがファイルに追記されたらその分を読み込んでJSONにして他に流します。</p>
<p>このログファイルはfluentdではない他の誰か、ApacheだったりRailsだったりします。<br>
この手のログファイルを書いていく流儀はいくつかありますが、ご存じの通りRailsやApacheは基本的に特定のパスにひたすら追記し続け、<br>
それでは肥大化しすぎるのでlogrotateなどと組み合わせて適宜rotateしていく運用が一般的でしょう。<br>
つまり、ログファイルをrenameし、ApacheやRailsなどにsignalで再起動して新しいファイルに書くわけです。</p>
<p>ところで、Windowsは誰かが開いているファイルをrenameすることはできないので通常このようなことは出来ません。<br>
やるにはファイルを開く際の<code>CreateFile</code>に<code>FILE_SHARE_DELETE</code>フラッグをつけてやる必要があります。<br>
そんなことやってる人いるのかというご指摘もあるとは思いますが、やる人はやっているようです。<br>
<a href="https://issues.apache.org/jira/browse/HADOOP-8564" class="external">https://issues.apache.org/jira/browse/HADOOP-8564</a></p>
<p>さて、RubyのWindowsの<code>open(2)</code>ラッパー、<code>rb_w32_open</code>/<code>rb_w32_wopen</code>はテキストモードの場合は<code>_open</code>/<code>_wopen</code>を使っており、<br>
これはおいておくとして、バイナリモードの場合は<code>CreateFile</code>を使っているものの現状<code>FILE_SHARE_DELETE</code>は渡していないので、<br>
なんらかの手段で<code>FILE_SHARE_DELETE</code>を付けられるようになりませんか。</p>
Ruby master - Feature #11216 (Closed): inode for Windows
https://redmine.ruby-lang.org/issues/11216
2015-06-03T13:32:22Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>現在WindowsではFile::Stat#inodeは常に0を返しますが、<br>
例えばあるlogrotate的な運用の行われるファイルを監視・開きっぱなしにして、追記されれば差分を読み、<br>
rotateされた場合は検出して最新のログを読む…というような場合に、rotateを検出する際にinodeは使われます。</p>
<p>WindowsにはGetFileInformationByHandle()というAPIで得られるBY_HANDLE_FILE_INFORMATION構造体のメンバ、<br>
nFileIndexHigh/Lowから64bitのファイルシステムuniqueなIDがあるので、これをinodeとして代用が可能です。</p>
<p>問題はこれをFile::Statにいれるにはもとのstruct stat.inoが16bitで小さすぎる点ですが、<br>
幸いstruct stati64にはアライメントの都合で隙間があるため、そこにつっこむことにしています。<br>
(devとudevが常に同じとか、uidやgidが常に0など無意味な値は他にもありますが)<br>
<a href="https://msdn.microsoft.com/ja-jp/library/ms350241%28v=vs.71%29.aspx" class="external">https://msdn.microsoft.com/ja-jp/library/ms350241%28v=vs.71%29.aspx</a></p>
<pre><code>diff --git a/file.c b/file.c
index ac99de6..761ff1f 100644
--- a/file.c
+++ b/file.c
@@ -548,7 +548,18 @@ rb_stat_dev_minor(VALUE self)
static VALUE
rb_stat_ino(VALUE self)
{
-#if SIZEOF_STRUCT_STAT_ST_INO > SIZEOF_LONG
+#ifdef _WIN32
+ struct stat *st = get_stat(self);
+ unsigned short *p2 = (unsigned short *)st;
+ unsigned int *p4 = (unsigned int *)st;
+ uint64_t r;
+ r = p2[2];
+ r <<= 16;
+ r |= p2[7];
+ r <<= 32;
+ r |= p4[5];
+ return ULL2NUM(r);
+#elif SIZEOF_STRUCT_STAT_ST_INO > SIZEOF_LONG
return ULL2NUM(get_stat(self)->st_ino);
#else
return ULONG2NUM(get_stat(self)->st_ino);
diff --git a/win32/win32.c b/win32/win32.c
index b23e9af..fcfc0a6 100644
--- a/win32/win32.c
+++ b/win32/win32.c
@@ -4975,6 +4975,48 @@ static time_t filetime_to_unixtime(const FILETIME *ft);
static WCHAR *name_for_stat(WCHAR *buf, const WCHAR *path);
static DWORD stati64_handle(HANDLE h, struct stati64 *st);
+/* License: Ruby's */
+static void
+stati64_set_inode(PBY_HANDLE_FILE_INFORMATION pinfo, struct stati64 *st)
+{
+ /* struct stati64 layout
+ *
+ * dev: 0-3
+ * ino: 4-5
+ * mode: 6-7
+ * nlink: 8-9
+ * uid: 10-11
+ * gid: 12-13
+ * _: 14-15
+ * rdev: 16-19
+ * _: 20-23
+ * size: 24-31
+ * atime: 32-39
+ * mtime: 40-47
+ * ctime: 48-55
+ *
+ */
+ unsigned short *p2 = (unsigned short *)st;
+ unsigned int *p4 = (unsigned int *)st;
+ DWORD high = pinfo->nFileIndexHigh;
+ p2[2] = high >> 16;
+ p2[7] = high & 0xFFFF;
+ p4[5] = pinfo->nFileIndexLow;
+}
+
+/* License: Ruby's */
+static DWORD
+stati64_set_inode_handle(HANDLE h, struct stati64 *st)
+{
+ BY_HANDLE_FILE_INFORMATION info;
+ DWORD attr = (DWORD)-1;
+
+ if (GetFileInformationByHandle(h, &info)) {
+ stati64_set_inode(&info, st);
+ }
+ return attr;
+}
+
#undef fstat
/* License: Ruby's */
int
@@ -5000,7 +5042,11 @@ rb_w32_fstati64(int fd, struct stati64 *st)
struct stat tmp;
int ret;
- if (GetEnvironmentVariableW(L"TZ", NULL, 0) == 0 && GetLastError() == ERROR_ENVVAR_NOT_FOUND) return _fstati64(fd, st);
+ if (GetEnvironmentVariableW(L"TZ", NULL, 0) == 0 && GetLastError() == ERROR_ENVVAR_NOT_FOUND) {
+ ret = _fstati64(fd, st);
+ stati64_set_inode_handle((HANDLE)_get_osfhandle(fd), st);
+ return ret;
+ }
ret = fstat(fd, &tmp);
if (ret) return ret;
@@ -5022,6 +5068,7 @@ stati64_handle(HANDLE h, struct stati64 *st)
st->st_mtime = filetime_to_unixtime(&info.ftLastWriteTime);
st->st_ctime = filetime_to_unixtime(&info.ftCreationTime);
st->st_nlink = info.nNumberOfLinks;
+ stati64_set_inode(&info, st);
attr = info.dwFileAttributes;
}
return attr;
</code></pre>
Ruby master - Misc #10553 (Closed): Ruby 2.2.0 release engeneering
https://redmine.ruby-lang.org/issues/10553
2014-11-28T13:01:41Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>For the management of release blockers.</p>
<p>Current Phase: feature freeze; changing feature is not allowed unless naruse explicitly allowed.<br>
Next release: RC1 (scheduled on 2014-12-1x)</p>
<p>Q: When ruby_2_2 branch is created?<br>
A: On RC1</p>
<p>Q: Can I add/change a feature?<br>
A: Get explicit permission from naruse</p>
Ruby master - Feature #9871 (Open): load a ruby library which doesn't have extension
https://redmine.ruby-lang.org/issues/9871
2014-05-28T10:07:07Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>拡張子のない Ruby スクリプトファイルを require する手段を提供しませんか。</p>
<p>Rubyスクリプトを拡張子無しで書くことはしばしばあります。<br>
例えば Unix のコマンドを Ruby で書くときがそうでしょう。</p>
<p>そのスクリプトの部品を将来再利用しそうなとき、場合によっては if $0 == <strong>FILE</strong> ハックを用いて、<br>
他のファイルから読み込まれたときはコマンドを起動を行わないようにするわけですが、<br>
拡張子がないとそもそも読み込めないので、まずファイル名を変えないといけません。</p>
<p>という具合で残念な感じなので、拡張子無しのファイルを読み込む手段を提供しませんか。<br>
require_relative は拡張子無しでも読める、辺りがいいと思うのですが。</p>
Ruby master - Feature #9816 (Assigned): 文字列内の数字を数値として比較するメソッド
https://redmine.ruby-lang.org/issues/9816
2014-05-08T09:37:26Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>文字列内の数字を数値として比較するメソッドを追加しませんか</p>
<p>そのような比較は一般的な用途としてはGUIシェルのファイラーが比較に用いており、<br>
Windows では StrCmpLogicalW が、OS X では NSString:compare:options:へのNSNumericSearch定数が提供されています。<br>
<a href="http://msdn.microsoft.com/en-us/library/windows/desktop/bb759947(v=vs.85).aspx" class="external">http://msdn.microsoft.com/en-us/library/windows/desktop/bb759947(v=vs.85).aspx</a><br>
<a href="https://developer.apple.com/library/mac/documentation/Cocoa/Reference/Foundation/Classes/NSString_Class/Reference/NSString.html#//apple_ref/c/econst/NSNumericSearch" class="external">https://developer.apple.com/library/mac/documentation/Cocoa/Reference/Foundation/Classes/NSString_Class/Reference/NSString.html#//apple_ref/c/econst/NSNumericSearch</a></p>
<p>上記のような処理自体はさほど難しいものではありませんが、Rubyレベルで実装すると大量のオブジェクトを作ってしまいます。<br>
例えば <code>Gem::Version.new("2.1.10".freeze)<=>Gem::Version.new("2.1.9".freeze)</code> は47個、<br>
<code>"2.1.10".freeze.split('.').map(&:to_i)<=>"2.1.9".freeze.split('.').map(&:to_i)</code> だと16個のオブジェクトを作ります。<br>
<code>"2.1.10".freeze.numericcmp"2.1.9".freeze</code> ならば、もちろんオブジェクトは一つも作りません。</p>
<p>なお、上記の例でも示唆していますが、本メソッドは Ruby のバージョン表記の TEENY が2桁になった場合の比較に用いることができます。</p>
<p>パッチは以下の通りです。<br>
なお、メソッド名は String#numericcmp としています。<br>
(String#casecmpを念頭に置いた)</p>
<pre><code>diff --git a/string.c b/string.c
index c589c80..66f667f 100644
--- a/string.c
+++ b/string.c
@@ -2569,6 +2569,131 @@ rb_str_casecmp(VALUE str1, VALUE str2)
return INT2FIX(-1);
}
+VALUE
+numerical_compare(const char **pp1, const char *p1end, const char **pp2, const char *p2end)
+{
+ const char *s1 = *pp1, *p1, *s2 = *pp2, *p2;
+ ptrdiff_t len1, len2;
+ int r;
+
+ while (s1 < p1end && *s1 == '0') s1++;
+ p1 = s1;
+ while (p1 < p1end && ISDIGIT(*p1)) p1++;
+ len1 = p1 - s1;
+
+ while (s2 < p2end && *s2 == '0') s2++;
+ p2 = s2;
+ while (p2 < p2end && ISDIGIT(*p2)) p2++;
+ len2 = p2 - s2;
+
+ if (len1 != len2) {
+ return INT2FIX(len1 < len2 ? -1 : 1);
+ }
+
+ r = memcmp(s1, s2, len1);
+ if (r) return r < 0 ? INT2FIX(-1) : INT2FIX(1);
+
+ len1 = s1 - *pp1;
+ len2 = s2 - *pp2;
+ if (len1 != len2) {
+ return INT2FIX(len1 < len2 ? -1 : 1);
+ }
+
+ *pp1 = p1;
+ *pp2 = p2;
+ return Qnil;
+}
+
+/*
+ * call-seq:
+ * str.numericcmp(other_str) -> -1, 0, +1 or nil
+ *
+ * Variant of <code>String#<=></code>, which considers digits in strings
+ * are numeric value..
+ *
+ * "a1".numericcmp("a1") #=> 0
+ * "aa".numericcmp("a1") #=> 1
+ * "a1".numericcmp("aa") #=> -1
+ * "a1".numericcmp("a01") #=> -1
+ * "2.1.2".numericcmp("2.1.10") #=> 1
+ */
+
+static VALUE
+rb_str_numericcmp(VALUE str1, VALUE str2)
+{
+ long len;
+ rb_encoding *enc;
+ const char *p1, *p1end, *p2, *p2end;
+
+ StringValue(str2);
+ enc = rb_enc_compatible(str1, str2);
+ if (!enc) {
+ return Qnil;
+ }
+
+ p1 = RSTRING_PTR(str1); p1end = RSTRING_END(str1);
+ p2 = RSTRING_PTR(str2); p2end = RSTRING_END(str2);
+ if (single_byte_optimizable(str1) && single_byte_optimizable(str2)) {
+ while (p1 < p1end && p2 < p2end) {
+ if (ISDIGIT(*p1)) {
+ if (ISDIGIT(*p2)) {
+ VALUE r = numerical_compare(&p1, p1end, &p2, p2end);
+ if (!NIL_P(r)) return r;
+ }
+ else {
+ return INT2FIX(-1);
+ }
+ }
+ else if (ISDIGIT(*p2)) {
+ return INT2FIX(1);
+ }
+ if (*p1 != *p2) return INT2FIX(*p1 < *p2 ? -1 : 1);
+ p1++;
+ p2++;
+ }
+ }
+ else {
+ while (p1 < p1end && p2 < p2end) {
+ int l1, c1 = rb_enc_ascget(p1, p1end, &l1, enc);
+ int l2, c2 = rb_enc_ascget(p2, p2end, &l2, enc);
+
+ if (0 <= c1 && 0 <= c2) {
+ if (ISDIGIT(*p1)) {
+ if (ISDIGIT(*p2)) {
+ VALUE r = numerical_compare(&p1, p1end, &p2, p2end);
+ if (!NIL_P(r)) return r;
+ }
+ else {
+ return INT2FIX(-1);
+ }
+ }
+ else if (ISDIGIT(*p2)) {
+ return INT2FIX(1);
+ }
+ if (*p1 != *p2) return INT2FIX(*p1 < *p2 ? -1 : 1);
+ p1++;
+ p2++;
+ }
+ else {
+ int r;
+ l1 = rb_enc_mbclen(p1, p1end, enc);
+ l2 = rb_enc_mbclen(p2, p2end, enc);
+ len = l1 < l2 ? l1 : l2;
+ r = memcmp(p1, p2, len);
+ if (r != 0)
+ return INT2FIX(r < 0 ? -1 : 1);
+ if (l1 != l2)
+ return INT2FIX(l1 < l2 ? -1 : 1);
+ }
+ p1 += l1;
+ p2 += l2;
+ }
+ }
+ if (RSTRING_LEN(str1) == RSTRING_LEN(str2)) return INT2FIX(0);
+ if (RSTRING_LEN(str1) > RSTRING_LEN(str2)) return INT2FIX(1);
+ return INT2FIX(-1);
+}
+
static long
rb_str_index(VALUE str, VALUE sub, long offset)
{
@@ -8721,6 +8846,7 @@ Init_String(void)
rb_define_method(rb_cString, "eql?", rb_str_eql, 1);
rb_define_method(rb_cString, "hash", rb_str_hash_m, 0);
rb_define_method(rb_cString, "casecmp", rb_str_casecmp, 1);
+ rb_define_method(rb_cString, "numericcmp", rb_str_numericcmp, 1);
rb_define_method(rb_cString, "+", rb_str_plus, 1);
rb_define_method(rb_cString, "*", rb_str_times, 1);
rb_define_method(rb_cString, "%", rb_str_format_m, 1);
diff --git a/test/ruby/test_string.rb b/test/ruby/test_string.rb
index 8366424..f9c788b 100644
--- a/test/ruby/test_string.rb
+++ b/test/ruby/test_string.rb
@@ -2104,6 +2104,29 @@ class TestString < Test::Unit::TestCase
assert_equal(1, "\u3042B".casecmp("\u3042a"))
end
+ def test_numericcmp
+ assert_equal(-1, "2.1.0".numericcmp("2.1.1"))
+ assert_equal(-1, "2.1.9".numericcmp("2.1.10"))
+ assert_equal( 0, "a1".numericcmp("a1"))
+ assert_equal( 1, "aa".numericcmp("a1"))
+ assert_equal(-1, "a1".numericcmp("aa"))
+ assert_equal(-1, "a1".numericcmp("a01"))
+ assert_equal(-1, "a0001".numericcmp("a00001"))
+ assert_equal( 0, "a1a".numericcmp("a1a"))
+ assert_equal( 1, "a1b".numericcmp("a1a"))
+ assert_equal(-1, "a9a".numericcmp("a10a"))
+ assert_equal( 1, "b".numericcmp("a"))
+ assert_equal( 0, "\u30421".numericcmp("\u30421"))
+ assert_equal( 1, "\u3042\u3042".numericcmp("\u30421"))
+ assert_equal(-1, "\u30421".numericcmp("\u3042\u3042"))
+ assert_equal(-1, "\u30421".numericcmp("\u304201"))
+ assert_equal(-1, "\u30420001".numericcmp("\u304200001"))
+ assert_equal( 0, "\u30421\u3042".numericcmp("\u30421\u3042"))
+ assert_equal( 1, "\u30421\u3044".numericcmp("\u30421\u3042"))
+ assert_equal(-1, "\u30429\u3042".numericcmp("\u304210\u3042"))
+ assert_equal( 1, "\u3044".numericcmp("\u3042"))
+ end
+
def test_upcase2
assert_equal("\u3042AB", "\u3042aB".upcase)
end
</code></pre>
Ruby master - Feature #9772 (Rejected): IO#statfs and File::Statfs
https://redmine.ruby-lang.org/issues/9772
2014-04-24T04:03:50Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>IO#statfs and File::Statfs を追加しませんか。<br>
(テストで statfs.f_type が必要だったのでとりあえず追加してしまっていますが)</p>
<p>statfs(2) は Unix 系 OS でそこそこ普及している、あるパスが属するファイルシステムについての情報を得るためのシステムコールです。<br>
OS によって多少差がありますが、だいたい以下の様な情報が得られます。</p>
<pre><code> struct statfs {
uint32_t f_type; /* type of filesystem */
uint64_t f_bsize; /* filesystem fragment size */
uint64_t f_blocks; /* total data blocks in filesystem */
uint64_t f_bfree; /* free blocks in filesystem */
int64_t f_bavail; /* free blocks avail to non-superuser */
uint64_t f_files; /* total file nodes in filesystem */
int64_t f_ffree; /* free nodes avail to non-superuser */
char f_fstypename[MFSNAMELEN]; /* filesystem type name */
};
</code></pre>
<p>f_type の値が OS 依存だったり、Linux 以外だとそもそもどれがどの値かきちんと定義されていないとか<br>
ツッコミどころの多い API ではあるのですが、他では得られない情報が得られます。</p>
<p>たとえば、以前から CRuby で使われている用途としては、あるファイルの乗っているファイルシステムが、<br>
HFS+ かどうかがわかります。言い換えると、ファイル名が正規化されているかどうかがわかります。<br>
ありがちな反論として、書き込めば正規化されるかわかるだろうというのがありえますが、<br>
目当てのファイルと同じディレクトリに書き込めるとは限りません。<br>
違うディレクトリだと別のファイルシステムがマウントされている可能性があります。</p>
<p>なお今回の用途は、SEEK_DATA/SEEK_HOLEができるか否かを、実際にやってみる以外の方法で知りたかった、というものです。<br>
(Rubyのテストなのにやってみて調べるではテストにあまりならない)</p>
Ruby master - Feature #9647 (Closed): File::Stat#birthtimeの追加
https://redmine.ruby-lang.org/issues/9647
2014-03-17T05:13:51Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>File::Stat#birthtimeを追加しませんか。</p>
<p>以下の様なシステムの stat(2) には st_birthtimespec があり、<br>
(ctime = change time ではなく) ファイルを作成した日時を得ることができます。<br>
<a href="http://netbsd.gw.com/cgi-bin/man-cgi?stat+2+NetBSD-current" class="external">http://netbsd.gw.com/cgi-bin/man-cgi?stat+2+NetBSD-current</a><br>
<a href="http://www.freebsd.org/cgi/man.cgi?query=stat&sektion=2&apropos=0&manpath=FreeBSD+10.0-RELEASE" class="external">http://www.freebsd.org/cgi/man.cgi?query=stat&sektion=2&apropos=0&manpath=FreeBSD+10.0-RELEASE</a><br>
<a href="https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/stat.2.html" class="external">https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/stat.2.html</a></p>
<p>また、Windowsはbirthtimeがあるがctimeがないという環境ですので、birthtimeはctimeを返します。<br>
<a href="http://msdn.microsoft.com/ja-jp/library/ms350241(v=vs.71).aspx" class="external">http://msdn.microsoft.com/ja-jp/library/ms350241(v=vs.71).aspx</a></p>
<p>今回のパッチではLinuxやOpenBSDなど、struct statにbirthtimeがない環境では、<br>
Windows同様ctimeを返すようにしています。<br>
(が、意味が違うからWindows以外ではNotImpErrorの方がいいかも)</p>
<p><a href="https://github.com/nurse/ruby/compare/ruby:trunk...birthtime" class="external">https://github.com/nurse/ruby/compare/ruby:trunk...birthtime</a></p>
Ruby master - Feature #8850 (Assigned): Convert Rational to decimal string
https://redmine.ruby-lang.org/issues/8850
2013-09-02T10:02:36Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>On Ruby 2.1.0, decimal literal is introduced.<br>
It generates Rational but it cannot easily convert to decimal string.<br>
You know, Rational#to_f is related to this but</p>
<ul>
<li>
<p>Float is not exact number<br>
** 0.123456789123456789r.to_f.to_s #=> "0.12345678912345678"</p>
</li>
<li>
<p>it can't handle recursive decimal<br>
** (151/13r).to_f.to_s #=> "11.615384615384615"</p>
</li>
<li>
<p>the method name<br>
** to_decimal<br>
** to_decimal_string<br>
** to_s(format: :decimal)<br>
** extend sprintf</p>
</li>
<li>
<p>how does it express recursive decimal<br>
** (151/13r).to_decimal_string #=> "11.615384..."<br>
** (151/13r).to_decimal_string #=> "11.(615384)"</p>
</li>
</ul>
<p>Example implementation is following.<br>
Its result is<br>
** 0.123456789123456789r.to_f.to_s #=> "0.123456789123456789"<br>
** (151/13r).to_f.to_s #=> "11.(615384)"</p>
<pre><code>class Rational
def to_decimal_string(base=10)
n = numerator
d = denominator
r, n = n.divmod d
str = r.to_s(base)
return str if n == 0
h = {}
str << '.'
n *= base
str.size.upto(Float::INFINITY) do |i|
r, n = n.divmod d
if n == 0
str << r.to_s(base)
break
elsif h.key? n
str[h[n], 0] = '('
str << ')'
break
else
str << r.to_s(base)
h[n] = i
n *= base
end
end
str
end
end
</code></pre>
Ruby master - Feature #8734 (Closed): irbに複数行履歴機能が欲しい
https://redmine.ruby-lang.org/issues/8734
2013-08-05T16:22:48Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>現在のirbはクラス定義やメソッド定義のような複数行にわたるものを書くと、<br>
実行結果は閉じるまで返ってこないのに、履歴は一行ごとにバラバラになってしまい、とても不便です。<br>
なので、zshのように、複数行をまとめて履歴として扱えるようになって欲しいです。<br>
参考: <a href="http://news.mynavi.jp/column/zsh/003/index.html" class="external">http://news.mynavi.jp/column/zsh/003/index.html</a></p>
Ruby master - Feature #8678 (Assigned): Allow invalid string to work with regexp
https://redmine.ruby-lang.org/issues/8678
2013-07-24T14:47:22Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Legacy Ruby 1.8 could regexp match with broken strings.<br>
People can find characters from binary data on the age.</p>
<p>After Ruby 1.9, Ruby raises Exception if it does regexp match with broken strings.<br>
So it became hard to work with character-wise regexp matching with binary data.</p>
<p>Following patch allows it with the constant Regexp::LOOSEENCODING.</p>
<p>commit eb0111ff7ae3f563ce201c4a5f724f121336d42d<br>
Author: NARUSE, Yui <a href="mailto:naruse@ruby-lang.org" class="email">naruse@ruby-lang.org</a><br>
Date: Mon Jul 22 05:37:44 2013 +0900</p>
<pre><code>* Regexp
* New constant:
* Regexp::ENCODINGLOOSE: declare execute matching even if the target string
is invalid byte sequence. [experimental]
</code></pre>
<p>diff --git a/NEWS b/NEWS<br>
index f5fe388..ade0b03 100644<br>
--- a/NEWS<br>
+++ b/NEWS<br>
@@ -35,6 +35,11 @@ with all sufficient information, see the ChangeLog file.</p>
<ul>
<li>misc
<ul>
<li>Mutex#owned? is no longer experimental.</li>
</ul>
</li>
</ul>
<p>+* Regexp</p>
<ul>
<li>
<ul>
<li>New constant:</li>
</ul>
</li>
<li>
<ul>
<li>Regexp::ENCODINGLOOSE: declare execute matching even if the target string</li>
</ul>
</li>
<li>
<pre><code> is invalid byte sequence. [experimental]
</code></pre>
</li>
<li>
</ul>
<ul>
<li>String
<ul>
<li>New methods:
<ul>
<li>String#scrub and String#scrub! verify and fix invalid byte sequence.<br>
diff --git a/re.c b/re.c<br>
index e5cc79d..230a2e0 100644<br>
--- a/re.c<br>
+++ b/re.c<br>
@@ -256,6 +256,7 @@ rb_memsearch(const void *x0, long m, const void *y0, long n, rb_encoding *enc)</li>
</ul>
</li>
</ul>
</li>
</ul>
<p>#define REG_LITERAL FL_USER5<br>
#define REG_ENCODING_NONE FL_USER6<br>
+#define REG_ENCODING_LOOSE FL_USER7</p>
<p>#define KCODE_FIXED FL_USER4</p>
<p>@@ -263,6 +264,7 @@ rb_memsearch(const void *x0, long m, const void *y0, long n, rb_encoding *enc)<br>
(ONIG_OPTION_IGNORECASE|ONIG_OPTION_MULTILINE|ONIG_OPTION_EXTEND)<br>
#define ARG_ENCODING_FIXED 16<br>
#define ARG_ENCODING_NONE 32<br>
+#define ARG_ENCODING_LOOSE 64</p>
<p>static int<br>
char_to_option(int c)<br>
@@ -1251,7 +1253,8 @@ rb_reg_prepare_enc(VALUE re, VALUE str, int warn)<br>
{<br>
rb_encoding *enc = 0;</p>
<ul>
<li>if (rb_enc_str_coderange(str) == ENC_CODERANGE_BROKEN) {</li>
</ul>
<ul>
<li>if (!(RBASIC(re)->flags & REG_ENCODING_LOOSE) &&</li>
<li>
<pre><code> rb_enc_str_coderange(str) == ENC_CODERANGE_BROKEN) {
rb_raise(rb_eArgError,
"invalid byte sequence in %s",
rb_enc_name(rb_enc_get(str)));
</code></pre>
</li>
</ul>
<p>@@ -2433,6 +2436,9 @@ rb_reg_initialize(VALUE obj, const char *s, long len, rb_encoding *enc,<br>
if (options & ARG_ENCODING_NONE) {<br>
re->basic.flags |= REG_ENCODING_NONE;<br>
}</p>
<ul>
<li>
<p>if (options & ARG_ENCODING_LOOSE) {</p>
</li>
<li>
<pre><code> re->basic.flags |= REG_ENCODING_LOOSE;
</code></pre>
</li>
<li>
<p>}</p>
<p>re->ptr = make_regexp(RSTRING_PTR(unescaped), RSTRING_LEN(unescaped), enc,<br>
options & ARG_REG_OPTION_MASK, err,<br>
@@ -3091,6 +3097,7 @@ rb_reg_options(VALUE re)<br>
options = RREGEXP(re)->ptr->options & ARG_REG_OPTION_MASK;<br>
if (RBASIC(re)->flags & KCODE_FIXED) options |= ARG_ENCODING_FIXED;<br>
if (RBASIC(re)->flags & REG_ENCODING_NONE) options |= ARG_ENCODING_NONE;</p>
</li>
<li>
<p>if (RBASIC(re)->flags & REG_ENCODING_LOOSE) options |= ARG_ENCODING_LOOSE;<br>
return options;<br>
}</p>
</li>
</ul>
<p>@@ -3579,6 +3586,8 @@ Init_Regexp(void)<br>
rb_define_const(rb_cRegexp, "FIXEDENCODING", INT2FIX(ARG_ENCODING_FIXED));<br>
/* see Regexp.options and Regexp.new */<br>
rb_define_const(rb_cRegexp, "NOENCODING", INT2FIX(ARG_ENCODING_NONE));</p>
<ul>
<li>
<p>/* see Regexp.options and Regexp.new */</p>
</li>
<li>
<p>rb_define_const(rb_cRegexp, "LOOSEENCODING", INT2FIX(ARG_ENCODING_LOOSE));</p>
<p>rb_global_variable(&reg_cache);</p>
</li>
</ul>
<p>diff --git a/string.c b/string.c<br>
index 1d784e3..caf0baf 100644<br>
--- a/string.c<br>
+++ b/string.c<br>
@@ -3970,7 +3970,7 @@ str_gsub(int argc, VALUE *argv, VALUE str, int bang)<br>
cp = sp;<br>
str_enc = STR_ENC_GET(str);<br>
rb_enc_associate(dest, str_enc);</p>
<ul>
<li>ENC_CODERANGE_SET(dest, rb_enc_asciicompat(str_enc) ? ENC_CODERANGE_7BIT : ENC_CODERANGE_VALID);</li>
</ul>
<ul>
<li>
<p>/<em>ENC_CODERANGE_SET(dest, rb_enc_asciicompat(str_enc) ? ENC_CODERANGE_7BIT : ENC_CODERANGE_VALID);</em>/</p>
<p>do {<br>
n++;<br>
diff --git a/test/ruby/test_regexp.rb b/test/ruby/test_regexp.rb<br>
index 11e86ec..b8f6897 100644<br>
--- a/test/ruby/test_regexp.rb<br>
+++ b/test/ruby/test_regexp.rb<br>
@@ -8,6 +8,10 @@ class TestRegexp < Test::Unit::TestCase<br>
$VERBOSE = nil<br>
end</p>
</li>
<li>
<p>def u(str)</p>
</li>
<li>
<p>str.dup.force_encoding(Encoding::UTF_8)</p>
</li>
<li>
<p>end</p>
</li>
<li>
<p>def teardown<br>
$VERBOSE = @verbose<br>
end<br>
@@ -958,6 +962,17 @@ class TestRegexp < Test::Unit::TestCase<br>
}<br>
end</p>
</li>
<li>
<p>def test_encoding_loose</p>
</li>
<li>
<p>str = u("\x80\xE3\x81\x82\x81")</p>
</li>
<li>
<p>assert_equal(0, Regexp.new(".", Regexp::LOOSEENCODING) =~ str)</p>
</li>
<li>
<p>assert_equal(1, Regexp.new(u('\p{Any}'), Regexp::LOOSEENCODING) =~ str)</p>
</li>
<li>
<p>assert_equal(1, Regexp.new("\u3042", Regexp::LOOSEENCODING) =~ str)</p>
</li>
<li>
<p>assert_equal(1, Regexp.new(u('\p{Hiragana}'), Regexp::LOOSEENCODING) =~ str)</p>
</li>
<li>
<p>assert_equal(0, Regexp.new(u('\A.\p{Hiragana}.\z'), Regexp::LOOSEENCODING) =~ str)</p>
</li>
<li>
<p>str = u("\xf1\x80\xE3\x81\x82\x81")</p>
</li>
<li>
<p>assert_equal(0, Regexp.new(u('\A..\p{Hiragana}.\z'), Regexp::LOOSEENCODING) =~ str)</p>
</li>
<li>
<p>end</p>
</li>
<li>
<a name="This-assertion-is-for-porting-x2-tests-in-testpypy-of-Onigmo"></a>
<h1 >This assertion is for porting x2() tests in testpy.py of Onigmo.<a href="#This-assertion-is-for-porting-x2-tests-in-testpypy-of-Onigmo" class="wiki-anchor">¶</a></h1>
<p>def assert_match_at(re, str, positions, msg = nil)<br>
re = Regexp.new(re) unless re.is_a?(Regexp)</p>
</li>
</ul>
Ruby master - Feature #8675 (Closed): Add Readline.point=(pos)
https://redmine.ruby-lang.org/issues/8675
2013-07-24T03:03:13Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>r42146 でのテスト修正で \A を入力しているのですが、rl_point を直接いじった方が正しい気がするので。</p>
<p>diff --git a/ext/readline/readline.c b/ext/readline/readline.c<br>
index 714b08c..03ab724 100644<br>
--- a/ext/readline/readline.c<br>
+++ b/ext/readline/readline.c<br>
@@ -808,6 +808,12 @@ readline_s_get_point(VALUE self)<br>
{<br>
return INT2NUM(rl_point);<br>
}<br>
+static VALUE<br>
+readline_s_set_point(VALUE self, VALUE pos)<br>
+{</p>
<ul>
<li>rl_point = NUM2INT(pos);</li>
<li>return pos;<br>
+}<br>
#else<br>
#define readline_s_get_point rb_f_notimplement<br>
#endif<br>
@@ -1761,6 +1767,8 @@ Init_readline()<br>
readline_s_get_line_buffer, 0);<br>
rb_define_singleton_method(mReadline, "point",<br>
readline_s_get_point, 0);</li>
<li>rb_define_singleton_method(mReadline, "point=",</li>
<li>
<pre><code> readline_s_set_point, 1);
</code></pre>
rb_define_singleton_method(mReadline, "set_screen_size",<br>
readline_s_set_screen_size, 2);<br>
rb_define_singleton_method(mReadline, "get_screen_size",</li>
</ul>
Ruby master - Feature #8539 (Closed): Unbundle ext/tk
https://redmine.ruby-lang.org/issues/8539
2013-06-18T00:25:36Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>How about unbundling ext/tk from Ruby repository?</p>
<p>ext/tk is a bundled extension library for GUI programming with tk.<br>
It is introduced in 1999 and long maintained with CRuby itself.</p>
<p>But nowadays its maintenance is not so active.<br>
Moreover ext/tk is not the de facto standard over Ruby GUI though it is bundled for 14 years.<br>
(maybe because tk is not de facto of GUI toolkit)<br>
GUI libraries for Ruby should compete in the wilds.</p>
<p>So I propose unbundling ext/tk.<br>
It should be another repository for example on github and people should install it as gem.</p>
<p>How do you think?</p>
Ruby master - Feature #8526 (Closed): gemify tk
https://redmine.ruby-lang.org/issues/8526
2013-06-14T10:22:28Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>tk を Ruby のリポジトリから外しませんか。</p>
<p>そもそも tk はサイズベースで言えば、約69MBのRubyリポジトリのうち、8.8MBを占める巨大なライブラリでかつ、<br>
ビルドに際して多くのライブラリを必要とするため、従前より周囲の目が厳しいライブラリでありましたが、<br>
これまでは精力的にメンテされている永井さんに敬意を払って、本格的な削除の提案はされて来ませんでした。</p>
<p>しかし、最近は活動が鈍り、#5199 <a class="issue tracker-4 status-5 priority-4 priority-default closed" title="Backport: Ruby 1.9.3 fails to compile with tcl/tk on s390x (Closed)" href="https://redmine.ruby-lang.org/issues/5465">#5465</a> <a class="issue tracker-1 status-6 priority-4 priority-default closed" title="Bug: compile error for ext/tk/tcltklib.c: ‘ruby_errinfo’ undeclared (Rejected)" href="https://redmine.ruby-lang.org/issues/5686">#5686</a> <a class="issue tracker-1 status-6 priority-4 priority-default closed" title="Bug: Tk::Scrollable の include が成功しない? (Rejected)" href="https://redmine.ruby-lang.org/issues/7000">#7000</a> <a class="issue tracker-1 status-6 priority-4 priority-default closed" title="Bug: Tkで,コマンドにforkを入れると,イベント実行時にクラッシュする (Rejected)" href="https://redmine.ruby-lang.org/issues/7884">#7884</a> <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: "require 'tk'" segfaults on 64-bit linux with Tk 8.6 (Closed)" href="https://redmine.ruby-lang.org/issues/8000">#8000</a> <a class="issue tracker-4 status-5 priority-4 priority-default closed" title="Backport: Tk::Canvas.create raise if type TkcItem (Closed)" href="https://redmine.ruby-lang.org/issues/8319">#8319</a> <a class="issue tracker-1 status-8 priority-4 priority-default closed" title="Bug: Can't build tcl/tk extensions after updating Debian/Ubuntu package (Third Party's Issue)" href="https://redmine.ruby-lang.org/issues/8435">#8435</a> が残タスクとして残っています。<br>
リリースマネージャとしては、このような状況下ではtkをRubyのリポジトリから外していただいて、<br>
gem等で永井さんのペースでメンテしていただくのがよいのではないでしょうか。</p>
Ruby master - Feature #8414 (Closed): String#scrub!
https://redmine.ruby-lang.org/issues/8414
2013-05-17T16:38:09Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>r40390 [Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Replacing ill-formed subsequencce (Closed)" href="https://redmine.ruby-lang.org/issues/6752">#6752</a>] で で追加した String#scrub ですが、破壊的メソッド版も追加していいでしょうか。<br>
lib/uri/common.rb いじっていて、欲しくなったので。</p>
Ruby master - Misc #8288 (Closed): Ruby 2.1.0 release engeneering
https://redmine.ruby-lang.org/issues/8288
2013-04-19T03:31:49Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>This is meta issue for releasing Ruby 2.1.0</p>
<p>see also <a href="https://bugs.ruby-lang.org/versions/27" class="external">https://bugs.ruby-lang.org/versions/27</a></p>
Backport200 - Backport #8266 (Closed): Backport r40216 (fiddle's mprotect)
https://redmine.ruby-lang.org/issues/8266
2013-04-14T23:23:40Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>r40216 を backport お願いします。<br>
とりあえず segv はしなくなります。</p>
Ruby master - Feature #8217 (Closed): OpenSSL::BN.new with integers
https://redmine.ruby-lang.org/issues/8217
2013-04-04T15:59:31Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Current OpenSSL::BN.new gets only strings, so users must do integer.to_s, it costs extra resource.<br>
Therefore I propose OpenSSL::BN.new to allow Fixnu/Bignum.</p>
<p>diff --git a/ext/openssl/ossl_bn.c b/ext/openssl/ossl_bn.c<br>
index 1038135..1f1ebba 100644<br>
--- a/ext/openssl/ossl_bn.c<br>
+++ b/ext/openssl/ossl_bn.c<br>
@@ -120,6 +120,44 @@ ossl_bn_initialize(int argc, VALUE *argv, VALUE self)<br>
base = NUM2INT(bs);<br>
}</p>
<ul>
<li>if (RB_TYPE_P(str, T_FIXNUM)) {</li>
<li>long i;</li>
<li>unsigned char <em>bin = (unsigned char</em>)ALLOC_N(long, 1);</li>
<li>long n = FIX2LONG(str);</li>
<li>unsigned long un = abs(n);</li>
<li>
<li>for (i = sizeof(VALUE) - 1; 0 <= i; i--) {</li>
<li>
<pre><code> bin[i] = un&0xff;
</code></pre>
</li>
<li>
<pre><code> un >>= 8;
</code></pre>
</li>
<li>}</li>
<li>
<li>GetBN(self, bn);</li>
<li>if (!BN_bin2bn(bin, sizeof(long), bn)) {</li>
<li>
<pre><code> ossl_raise(eBNError, NULL);
</code></pre>
</li>
<li>}</li>
<li>if (n < 0) BN_set_negative(bn, 1);</li>
<li>return self;</li>
<li>}</li>
<li>else if (RB_TYPE_P(str, T_BIGNUM)) {</li>
<li>long i, j;</li>
<li>BDIGIT *ds = RBIGNUM_DIGITS(str);</li>
<li>unsigned char <em>bin = (unsigned char</em>)ALLOC_N(BDIGIT, RBIGNUM_LEN(str));</li>
<li>
<li>for (i = 0; RBIGNUM_LEN(str) > i; i++) {</li>
<li>
<pre><code> BDIGIT v = ds[i];
</code></pre>
</li>
<li>
<pre><code> for (j = sizeof(BDIGIT) - 1; 0 <= j; j--) {
</code></pre>
</li>
<li>
<pre><code> bin[(RBIGNUM_LEN(str)-1-i)*sizeof(BDIGIT)+j] = v&0xff;
</code></pre>
</li>
<li>
<pre><code> v >>= 8;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>}</li>
<li>
<li>GetBN(self, bn);</li>
<li>if (!BN_bin2bn(bin, sizeof(BDIGIT)*RBIGNUM_LEN(str), bn)) {</li>
<li>
<pre><code> ossl_raise(eBNError, NULL);
</code></pre>
</li>
<li>}</li>
<li>if (!RBIGNUM_SIGN(str)) BN_set_negative(bn, 1);</li>
<li>return self;</li>
<li>}<br>
if (RTEST(rb_obj_is_kind_of(str, cBN))) {<br>
BIGNUM *other;</li>
</ul>
<p>diff --git a/test/openssl/test_bn.rb b/test/openssl/test_bn.rb<br>
index af1c72c..758cc54 100644<br>
--- a/test/openssl/test_bn.rb<br>
+++ b/test/openssl/test_bn.rb<br>
@@ -8,6 +8,13 @@ class OpenSSL::TestBN < Test::Unit::TestCase<br>
end</p>
<p>def test_integer_to_bn</p>
<ul>
<li>assert_equal(999.to_bn, OpenSSL::BN.new(999))</li>
<li>assert_equal((2 ** 107 - 1).to_bn, OpenSSL::BN.new(2 ** 107 - 1))</li>
<li>assert_equal(-999.to_bn, OpenSSL::BN.new(-999))</li>
<li>assert_equal((-(2 ** 107 - 1)).to_bn, OpenSSL::BN.new(-(2 ** 107 - 1)))</li>
<li>end</li>
<li>
<li>def test_integer_str_to_bn<br>
assert_equal(999.to_bn, OpenSSL::BN.new(999.to_s(16), 16))<br>
assert_equal((2 ** 107 - 1).to_bn, OpenSSL::BN.new((2 ** 107 - 1).to_s(16), 16))<br>
end</li>
</ul>
Backport193 - Backport #8076 (Closed): Lookbehind assertion fails with /m mode enabled
https://redmine.ruby-lang.org/issues/8076
2013-03-11T19:52:38Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Lookbehind assertions fail if they are longer than one character, and if dotall mode is set.</p>
<p>irb(main):001:0> "foo" =~ /(?<=f).<em>/m<br>
=> 1<br>
irb(main):002:0> "foo" =~ /(?<=fo).</em>/m<br>
=> nil</p>
<p>The latter should have matched the "o". This only seems to happen with dotall mode turned on (dot matches newline); without it, everything is OK:</p>
<p>irb(main):003:0> "foo" =~ /(?<=f).<em>/<br>
=> 1<br>
irb(main):004:0> "foo" =~ /(?<=fo).</em>/<br>
=> 2</p>
Backport193 - Backport #7896 (Closed): Can't test rb_iter_break with extensions
https://redmine.ruby-lang.org/issues/7896
2013-02-21T12:05:29Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>If you test rb_iter_break() with some extension library like trunk's ext/-test-/iter/break.c, it will cause SEGV on 1.9.3.<br>
It won't happen on 2.0 because r34369 is accidentally fix it with below patch.</p>
<p>I noticed this because RubySpec added a test for rb_spec_iter (4db31b04954118e66ac1d6353ebf4106cb2b419b) and hit this.</p>
<h1>% svn di<br>
Index: vm.c</h1>
<p>--- vm.c (revision 39346)<br>
+++ vm.c (working copy)<br>
@@ -1370,6 +1370,7 @@<br>
*th->cfp->sp++ = (GET_THROWOBJ_VAL(err));<br>
#endif<br>
}</p>
<ul>
<li>
<pre><code> th->state = 0;
th->errinfo = Qnil;
goto vm_loop_start;
}
</code></pre>
</li>
</ul>
Backport193 - Backport #7587 (Closed): r35924
https://redmine.ruby-lang.org/issues/7587
2012-12-18T19:56:48Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>make -j したときにこけるので r35924 を backport してください</p>
Backport193 - Backport #7169 (Closed): r37169
https://redmine.ruby-lang.org/issues/7169
2012-10-16T09:43:05Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>以下をマージお願いします。</p>
<p>commit 78e3185a20d9307c98fecca6d3bacfb8f2c271b8<br>
Author: naruse <a href="mailto:naruse@b2dd03c8-39d4-4d8f-98ff-823fe69b080e" class="email">naruse@b2dd03c8-39d4-4d8f-98ff-823fe69b080e</a><br>
Date: Sat Oct 13 01:03:41 2012</p>
<pre><code>use tty(1) to check if /dev/tty is usable or not
</code></pre>
<p><a href="http://svn.ruby-lang.org/cgi-bin/viewvc.cgi?revision=37169&view=revision" class="external">http://svn.ruby-lang.org/cgi-bin/viewvc.cgi?revision=37169&view=revision</a></p>
Ruby master - Feature #6936 (Closed): Forbid singleton class and instance variabls for float
https://redmine.ruby-lang.org/issues/6936
2012-08-27T08:12:45Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>[Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Introduce Flonum technique to speedup floating computation on th 64bit environment (Closed)" href="https://redmine.ruby-lang.org/issues/6763">#6763</a>] などで議論されていた flonum が r36798 でが入ったわけですが、</p>
<ol>
<li>Float のオブジェクトID の仕様が変更</li>
<li>flonum な float に特異メソッドが追加不可</li>
<li>flonum な float に特異クラスが作成不可</li>
<li>flonum な float は同じ値同士でインスタンス変数が共有される</li>
</ol>
<p>といった非互換が存在します。<br>
もっとも、1. は通常意識するはずのないところですし、2. は元から禁止されています。<br>
気になるのは 3. と 4. で、これは 1.9.3 と挙動が異なるだけでなく、<br>
32bit 環境での 2.0 や、64bit環境の flonum でない float オブジェクトとも挙動が異なります。</p>
<p>実際問題として実害はないような気もしますが、このような違いが極めて実装上の問題で、<br>
Ruby 上から見えないところに存在するのは気持ち悪く感じます。</p>
<p>よって、以下のようにするとよいのではないでしょうか。</p>
<ul>
<li>flonum でない float でも特異クラスの作成を禁止</li>
<li>float へのインスタンス変数作成を禁止</li>
</ul>
<p>後者の具体的手法はいくつかあると思いますが、即値は最初から frozen にしておくとかもありかなと思っています。</p>
<p>話を発散させると、この話は true, false, nil, Fixnum, Symbol のような即値から、<br>
Bignum や Time のような immutable っぽいオブジェクトにも当てはまる気がしています。</p>
Ruby master - Feature #6910 (Rejected): Loading syck's broken yaml with psych
https://redmine.ruby-lang.org/issues/6910
2012-08-23T11:04:21Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>You know, syck outputs wrong yaml.<br>
For example, syck works as following:</p>
<p>ruby-1.9.2 > ["\u3042",Time.at(0).to_s].to_yaml<br>
=> "--- \n- "\xE3\x81\x82"\n- 1970-01-01 09:00:00 +09:00\n"</p>
<p>It should be</p>
<p>ruby-1.9.3 > ["\u3042",Time.at(0).to_s].to_yaml<br>
=> "---\n- あ\n- '1970-01-01 09:00:00 +0900'\n"</p>
<p>syck's dump of Unicode string is interpreted as "\u00E3\u0081\u0082".<br>
syck's dump of Time like string is interpreted as Time.<br>
It is hard to migrate old data to new and correct data, so it is useful if psych has a such compatibility option.</p>
Ruby master - Feature #6767 (Closed): Utility method to get a duplicated string whose encoding is...
https://redmine.ruby-lang.org/issues/6767
2012-07-22T03:20:31Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>ある String を ASCII-8BIT にしたいことはしばしばあります。<br>
それだけならばまだ force_encoding しろよという話なのですが、<br>
<a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Bitwise string operations (Rejected)" href="https://redmine.ruby-lang.org/issues/6361">#6361</a> の例のように、バイナリ文字列にさらにバイナリ文字列を結合していく場合、<br>
毎行毎行 force_encoding を書いていくのにはつらいものがあります。</p>
<p>解決案としては、<br>
(1) バイナリリテラルの導入<br>
(2) dup.force_encoding(Encoding::ASCII_8BIT) する短いメソッドを追加<br>
(3) ASCII-8BIT に他のエンコーディングの文字列を結合した場合は暗黙に force_encoding<br>
が考えられます。</p>
<p>しかし、(1) は文法拡張なのでハードルが高く、(3) は方々で議論になっている通りです。<br>
よって、(2) が妥当ではないかと思います。</p>
<p>名前をまつもとさん提案の String#b としたパッチを以下の通り添付します。</p>
<p>diff --git a/string.c b/string.c<br>
index d038835..76cbc36 100644<br>
--- a/string.c<br>
+++ b/string.c<br>
@@ -601,7 +601,7 @@ rb_str_export_to_enc(VALUE str, rb_encoding *enc)<br>
}</p>
<p>static VALUE<br>
-str_replace_shared(VALUE str2, VALUE str)<br>
+str_replace_shared_without_enc(VALUE str2, VALUE str)<br>
{<br>
if (RSTRING_LEN(str) <= RSTRING_EMBED_LEN_MAX) {<br>
STR_SET_EMBED(str2);<br>
@@ -616,8 +616,14 @@ str_replace_shared(VALUE str2, VALUE str)<br>
RSTRING(str2)->as.heap.aux.shared = str;<br>
FL_SET(str2, ELTS_SHARED);<br>
}</p>
<ul>
<li>rb_enc_cr_str_exact_copy(str2, str);</li>
</ul>
<ul>
<li>return str2;<br>
+}</li>
</ul>
<p>+static VALUE<br>
+str_replace_shared(VALUE str2, VALUE str)<br>
+{</p>
<ul>
<li>str_replace_shared_without_enc(str2, str);</li>
<li>rb_enc_cr_str_exact_copy(str2, str);<br>
return str2;<br>
}</li>
</ul>
<p>@@ -7340,6 +7346,23 @@ rb_str_force_encoding(VALUE str, VALUE enc)</p>
<p>/*</p>
<ul>
<li>call-seq:</li>
</ul>
<ul>
<li>
<ul>
<li>
<pre><code>str.b -> str
</code></pre>
</li>
</ul>
</li>
<li>
<ul>
<li>
</ul>
</li>
<li>
<ul>
<li>Returns a copied string whose encoding is ASCII-8BIT.</li>
</ul>
</li>
<li>*/</li>
<li>
</ul>
<p>+static VALUE<br>
+rb_str_b(VALUE str)<br>
+{</p>
<ul>
<li>VALUE str2 = str_alloc(rb_cString);</li>
<li>str_replace_shared_without_enc(str2, str);</li>
<li>OBJ_INFECT(str2, str);</li>
<li>ENC_CODERANGE_SET(str2, ENC_CODERANGE_VALID);</li>
<li>return str2;<br>
+}</li>
<li>
</ul>
<p>+/*</p>
<ul>
<li>
<ul>
<li>
<p>call-seq:</p>
</li>
<li>
<pre><code>str.valid_encoding? -> true or false
</code></pre>
</li>
<li>
<li>
<p>Returns true for a string which encoded correctly.<br>
@@ -7969,6 +7992,7 @@ Init_String(void)</p>
<p>rb_define_method(rb_cString, "encoding", rb_obj_encoding, 0); /* in encoding.c */<br>
rb_define_method(rb_cString, "force_encoding", rb_str_force_encoding, 1);</p>
</li>
</ul>
</li>
<li>rb_define_method(rb_cString, "b", rb_str_b, 0);<br>
rb_define_method(rb_cString, "valid_encoding?", rb_str_valid_encoding_p, 0);<br>
rb_define_method(rb_cString, "ascii_only?", rb_str_is_ascii_only_p, 0);</li>
</ul>
<p>diff --git a/test/ruby/test_m17n.rb b/test/ruby/test_m17n.rb<br>
index dfcaa94..3a4bca7 100644<br>
--- a/test/ruby/test_m17n.rb<br>
+++ b/test/ruby/test_m17n.rb<br>
@@ -1469,4 +1469,14 @@ class TestM17N < Test::Unit::TestCase<br>
yield(*strs)<br>
end<br>
end<br>
+</p>
<ul>
<li>def test_str_b</li>
<li>s = "\u3042"</li>
<li>assert_equal(a("\xE3\x81\x82"), s.b)</li>
<li>assert_equal(Encoding::ASCII_8BIT, s.b.encoding)</li>
<li>s.taint</li>
<li>assert_equal(true, s.b.tainted?)</li>
<li>s.untrust</li>
<li>assert_equal(true, s.b.untrusted?)</li>
<li>end<br>
end</li>
</ul>
Ruby master - Feature #6752 (Closed): Replacing ill-formed subsequencce
https://redmine.ruby-lang.org/issues/6752
2012-07-19T11:42:58Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
== 概要<br>
Stringになんらかの理由で不正なバイト列が含まれている時に、それを置換文字で置き換えたい。</p>
<p>== ユースケース<br>
実際に確認されているユースケースは以下の通りです。</p>
<ul>
<li>
<p>twitterのtitle</p>
</li>
<li>
<p>IRCのログ</p>
</li>
<li>
<p>ニコニコ動画の API</p>
</li>
<li>
<p>Webクローリング<br>
これらの不正なバイト列の生成過程は、おそらく、バイト単位で文字列を切り詰めた時に末尾が切れて、<br>
末尾がおかしい不正な文字列が作られます。(前二者)<br>
これをコンテナに入れたり結合することによって、途中にも混ざった文字列が作られます。(後二者)</p>
</li>
<li>
<p><a href="https://twitter.com/takahashim/status/18974040397" class="external">https://twitter.com/takahashim/status/18974040397</a></p>
</li>
<li>
<p><a href="https://twitter.com/n0kada/status/215674740705210368" class="external">https://twitter.com/n0kada/status/215674740705210368</a></p>
</li>
<li>
<p><a href="https://twitter.com/n0kada/status/215686490070585346" class="external">https://twitter.com/n0kada/status/215686490070585346</a></p>
</li>
<li>
<p><a href="https://twitter.com/hajimehoshi/status/215671146769682432" class="external">https://twitter.com/hajimehoshi/status/215671146769682432</a></p>
</li>
<li>
<p><a href="http://po-ru.com/diary/fixing-invalid-utf-8-in-ruby-revisited/" class="external">http://po-ru.com/diary/fixing-invalid-utf-8-in-ruby-revisited/</a></p>
</li>
<li>
<p><a href="http://stackoverflow.com/questions/2982677/ruby-1-9-invalid-byte-sequence-in-utf-8" class="external">http://stackoverflow.com/questions/2982677/ruby-1-9-invalid-byte-sequence-in-utf-8</a></p>
</li>
</ul>
<p>== 必要な引数: 置換文字<br>
省略可能、String。<br>
デフォルトは、Unicode系ならU+FFFD、それ以外では「?」。<br>
デフォルトが空文字でない理由は、削除してしまうことで、従来は存在しなかったトークンを作れてしまい、<br>
上位のレイヤーの脆弱性に繋がるからです。<br>
<a href="http://unicode.org/reports/tr36/#UTF-8_Exploit" class="external">http://unicode.org/reports/tr36/#UTF-8_Exploit</a></p>
<p>== API<br>
--- str.encode(str.encoding, invalid: replace, [replace: "〓"])</p>
<ul>
<li>CSI的じゃなくて気持ち悪い</li>
<li>iconv でできるのは glibc iconv か GNU libiconv に //IGNORE つけた時で他はできない</li>
<li>実装上のメリットは後述の通り、直感に反してあまりない(と思う)</li>
</ul>
<p>== 別メソッド</p>
<ul>
<li>新しいメソッドである</li>
<li>fix/repair invalid/illegal bytes/sequence あたりの名前か</li>
</ul>
<p>== 実装<br>
=== 鬼車ベース<br>
int ret = rb_enc_precise_mbclen(p, e, enc); して、<br>
MBCLEN_INVALID_P(ret) が真な時、何バイト目が不正なのかわからないのが微妙。<br>
ONIGENC_CONSTRUCT_MBCLEN_INVALID() がバイト数を取らないのが原因なので、<br>
鬼車のエンコーディングモジュール全てに影響してしまうため、修正困難。<br>
不正なバイトはほとんど存在しないと仮定して、効率を犠牲にすれば回避は可能。</p>
<p>=== transcodeベース<br>
UCS正規化なglibc iconv, GNU libiconv, Perl Encodeなどと違って、<br>
CSIなtranscodeでは、自分自身に変換する場合、<br>
エンコーディングごとに「何もしない」変換モジュールを用意しないといけない。</p>
<p>とりあえず鬼車ベースのコンセプト実装とテストを添付しておきます。</p>
<p>diff --git a/string.c b/string.c<br>
index d038835..4808f15 100644<br>
--- a/string.c<br>
+++ b/string.c<br>
@@ -7426,6 +7426,199 @@ rb_str_ellipsize(VALUE str, long len)<br>
return ret;<br>
}</p>
<p>+/*</p>
<ul>
<li>
<ul>
<li>call-seq:</li>
</ul>
</li>
<li>
<ul>
<li>str.fix_invalid -> new_str</li>
</ul>
</li>
<li>
<ul>
<li>
</ul>
</li>
<li>
<ul>
<li>If the string is well-formed, it returns self.</li>
</ul>
</li>
<li>
<ul>
<li>If the string has invalid byte sequence, repair it with given replacement</li>
</ul>
</li>
<li>
<ul>
<li>character.</li>
</ul>
</li>
<li>*/<br>
+VALUE<br>
+rb_str_fix_invalid(VALUE str)<br>
+{</li>
<li>int cr = ENC_CODERANGE(str);</li>
<li>rb_encoding *enc;</li>
<li>if (cr == ENC_CODERANGE_7BIT || cr == ENC_CODERANGE_VALID)</li>
<li>return rb_str_dup(str);</li>
<li>
<li>enc = STR_ENC_GET(str);</li>
<li>if (rb_enc_asciicompat(enc)) {</li>
<li>const char *p = RSTRING_PTR(str);</li>
<li>const char *e = RSTRING_END(str);</li>
<li>const char *p1 = p;</li>
<li>/* 10 should be enough for the usual use case,</li>
<li>
<ul>
<li>fixing a wrongly chopped character at the end of the string</li>
</ul>
</li>
<li>*/</li>
<li>long room = 10;</li>
<li>VALUE buf = rb_str_buf_new(RSTRING_LEN(str) + room);</li>
<li>const char *rep;</li>
<li>if (enc == rb_utf8_encoding())</li>
<li>
<pre><code> rep = "\xEF\xBF\xBD";
</code></pre>
</li>
<li>else</li>
<li>
<pre><code> rep = "?";
</code></pre>
</li>
<li>cr = ENC_CODERANGE_7BIT;</li>
<li>
<li>p = search_nonascii(p, e);</li>
<li>if (!p) {</li>
<li>
<pre><code> p = e;
</code></pre>
</li>
<li>}</li>
<li>while (p < e) {</li>
<li>
<pre><code> int ret = rb_enc_precise_mbclen(p, e, enc);
</code></pre>
</li>
<li>
<pre><code> if (MBCLEN_CHARFOUND_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> if ((unsigned char)*p > 127) cr = ENC_CODERANGE_VALID;
</code></pre>
</li>
<li>
<pre><code> p += MBCLEN_CHARFOUND_LEN(ret);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else if (MBCLEN_INVALID_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> const char *q;
</code></pre>
</li>
<li>
<pre><code> long clen = rb_enc_mbmaxlen(enc);
</code></pre>
</li>
<li>
<pre><code> if (p > p1) rb_str_buf_cat(buf, p1, p - p1);
</code></pre>
</li>
<li>
<pre><code> q = RSTRING_END(buf);
</code></pre>
</li>
<li>
<li>
<pre><code> if (e - p < clen) clen = e - p;
</code></pre>
</li>
<li>
<pre><code> if (clen < 3) {
</code></pre>
</li>
<li>
<pre><code> clen = 1;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else {
</code></pre>
</li>
<li>
<pre><code> long len = RSTRING_LEN(buf);
</code></pre>
</li>
<li>
<pre><code> clen--;
</code></pre>
</li>
<li>
<pre><code> rb_str_buf_cat(buf, p, clen);
</code></pre>
</li>
<li>
<pre><code> for (; clen > 1; clen--) {
</code></pre>
</li>
<li>
<pre><code> ret = rb_enc_precise_mbclen(q, q + clen, enc);
</code></pre>
</li>
<li>
<pre><code> if (MBCLEN_NEEDMORE_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else if (MBCLEN_INVALID_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> continue;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else {
</code></pre>
</li>
<li>
<pre><code> rb_bug("shouldn't reach here '%s'", q);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> rb_str_set_len(buf, len);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> p += clen;
</code></pre>
</li>
<li>
<pre><code> p1 = p;
</code></pre>
</li>
<li>
<pre><code> rb_str_buf_cat2(buf, rep);
</code></pre>
</li>
<li>
<pre><code> p = search_nonascii(p, e);
</code></pre>
</li>
<li>
<pre><code> if (!p) {
</code></pre>
</li>
<li>
<pre><code> p = e;
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else if (MBCLEN_NEEDMORE_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else {
</code></pre>
</li>
<li>
<pre><code> rb_bug("shouldn't reach here");
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>}</li>
<li>if (p1 < p) {</li>
<li>
<pre><code> rb_str_buf_cat(buf, p1, p - p1);
</code></pre>
</li>
<li>}</li>
<li>if (p < e) {</li>
<li>
<pre><code> rb_str_buf_cat2(buf, rep);
</code></pre>
</li>
<li>
<pre><code> cr = ENC_CODERANGE_VALID;
</code></pre>
</li>
<li>}</li>
<li>ENCODING_CODERANGE_SET(buf, rb_enc_to_index(enc), cr);</li>
<li>return buf;</li>
<li>}</li>
<li>else if (rb_enc_dummy_p(enc)) {</li>
<li>return rb_str_dup(str);</li>
<li>}</li>
<li>else {</li>
<li>/* ASCII incompatible */</li>
<li>const char *p = RSTRING_PTR(str);</li>
<li>const char *e = RSTRING_END(str);</li>
<li>const char *p1 = p;</li>
<li>/* 10 should be enough for the usual use case,</li>
<li>
<ul>
<li>fixing a wrongly chopped character at the end of the string</li>
</ul>
</li>
<li>*/</li>
<li>long room = 10;</li>
<li>VALUE buf = rb_str_buf_new(RSTRING_LEN(str) + room);</li>
<li>const char *rep;</li>
<li>long mbminlen = rb_enc_mbminlen(enc);</li>
<li>static rb_encoding *utf16be;</li>
<li>static rb_encoding *utf16le;</li>
<li>static rb_encoding *utf32be;</li>
<li>static rb_encoding *utf32le;</li>
<li>if (!utf16be) {</li>
<li>
<pre><code> utf16be = rb_enc_find("UTF-16BE");
</code></pre>
</li>
<li>
<pre><code> utf16le = rb_enc_find("UTF-16LE");
</code></pre>
</li>
<li>
<pre><code> utf32be = rb_enc_find("UTF-32BE");
</code></pre>
</li>
<li>
<pre><code> utf32le = rb_enc_find("UTF-32LE");
</code></pre>
</li>
<li>}</li>
<li>if (enc == utf16be) {</li>
<li>
<pre><code> rep = "\xFF\xFD";
</code></pre>
</li>
<li>}</li>
<li>else if (enc == utf16le) {</li>
<li>
<pre><code> rep = "\xFD\xFF";
</code></pre>
</li>
<li>}</li>
<li>else if (enc == utf32be) {</li>
<li>
<pre><code> rep = "\x00\x00\xFF\xFD";
</code></pre>
</li>
<li>}</li>
<li>else if (enc == utf32le) {</li>
<li>
<pre><code> rep = "\xFD\xFF\x00\x00";
</code></pre>
</li>
<li>}</li>
<li>else {</li>
<li>
<pre><code> rep = "?";
</code></pre>
</li>
<li>}</li>
<li>
<li>while (p < e) {</li>
<li>
<pre><code> int ret = rb_enc_precise_mbclen(p, e, enc);
</code></pre>
</li>
<li>
<pre><code> if (MBCLEN_CHARFOUND_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> p += MBCLEN_CHARFOUND_LEN(ret);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else if (MBCLEN_INVALID_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> const char *q;
</code></pre>
</li>
<li>
<pre><code> long clen = rb_enc_mbmaxlen(enc);
</code></pre>
</li>
<li>
<pre><code> if (p > p1) rb_str_buf_cat(buf, p1, p - p1);
</code></pre>
</li>
<li>
<pre><code> q = RSTRING_END(buf);
</code></pre>
</li>
<li>
<li>
<pre><code> if (e - p < clen) clen = e - p;
</code></pre>
</li>
<li>
<pre><code> if (clen < mbminlen * 3) {
</code></pre>
</li>
<li>
<pre><code> clen = mbminlen;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else {
</code></pre>
</li>
<li>
<pre><code> long len = RSTRING_LEN(buf);
</code></pre>
</li>
<li>
<pre><code> clen -= mbminlen;
</code></pre>
</li>
<li>
<pre><code> rb_str_buf_cat(buf, p, clen);
</code></pre>
</li>
<li>
<pre><code> for (; clen > mbminlen; clen-=mbminlen) {
</code></pre>
</li>
<li>
<pre><code> ret = rb_enc_precise_mbclen(q, q + clen, enc);
</code></pre>
</li>
<li>
<pre><code> if (MBCLEN_NEEDMORE_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else if (MBCLEN_INVALID_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> continue;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else {
</code></pre>
</li>
<li>
<pre><code> rb_bug("shouldn't reach here '%s'", q);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> rb_str_set_len(buf, len);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> p += clen;
</code></pre>
</li>
<li>
<pre><code> p1 = p;
</code></pre>
</li>
<li>
<pre><code> rb_str_buf_cat2(buf, rep);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else if (MBCLEN_NEEDMORE_P(ret)) {
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else {
</code></pre>
</li>
<li>
<pre><code> rb_bug("shouldn't reach here");
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>}</li>
<li>if (p1 < p) {</li>
<li>
<pre><code> rb_str_buf_cat(buf, p1, p - p1);
</code></pre>
</li>
<li>}</li>
<li>if (p < e) {</li>
<li>
<pre><code> rb_str_buf_cat2(buf, rep);
</code></pre>
</li>
<li>}</li>
<li>ENCODING_CODERANGE_SET(buf, rb_enc_to_index(enc), ENC_CODERANGE_VALID);</li>
<li>return buf;</li>
<li>}<br>
+}</li>
<li>
</ul>
<p>/**********************************************************************</p>
<ul>
<li>Document-class: Symbol</li>
<li>
</ul>
<p>@@ -7882,6 +8075,7 @@ Init_String(void)<br>
rb_define_method(rb_cString, "getbyte", rb_str_getbyte, 1);<br>
rb_define_method(rb_cString, "setbyte", rb_str_setbyte, 2);<br>
rb_define_method(rb_cString, "byteslice", rb_str_byteslice, -1);</p>
<ul>
<li>
<p>rb_define_method(rb_cString, "fix_invalid", rb_str_fix_invalid, 0);</p>
<p>rb_define_method(rb_cString, "to_i", rb_str_to_i, -1);<br>
rb_define_method(rb_cString, "to_f", rb_str_to_f, 0);<br>
diff --git a/test/ruby/test_string.rb b/test/ruby/test_string.rb<br>
index 47f349c..2b0cfeb 100644<br>
--- a/test/ruby/test_string.rb<br>
+++ b/test/ruby/test_string.rb<br>
@@ -2031,6 +2031,29 @@ class TestString < Test::Unit::TestCase</p>
<p>assert_equal(u("\x82")+("\u3042"*9), ("\u3042"*10).byteslice(2, 28))<br>
end</p>
</li>
<li>
<li>
<p>def test_fix_invalid</p>
</li>
<li>
<p>assert_equal("\uFFFD\uFFFD\uFFFD", "\x80\x80\x80".fix_invalid)</p>
</li>
<li>
<p>assert_equal("\uFFFDA", "\xF4\x80\x80A".fix_invalid)</p>
</li>
<li>
<li>
<a name="exapmles-in-Unicode-610-D93b"></a>
<h1 >exapmles in Unicode 6.1.0 D93b<a href="#exapmles-in-Unicode-610-D93b" class="wiki-anchor">¶</a></h1>
</li>
<li>
<p>assert_equal("\x41\uFFFD\uFFFD\x41\uFFFD\x41",</p>
</li>
<li>
<pre><code> "\x41\xC0\xAF\x41\xF4\x80\x80\x41".fix_invalid)
</code></pre>
</li>
<li>
<p>assert_equal("\x41\uFFFD\uFFFD\uFFFD\x41",</p>
</li>
<li>
<pre><code> "\x41\xE0\x9F\x80\x41".fix_invalid)
</code></pre>
</li>
<li>
<p>assert_equal("\u0061\uFFFD\uFFFD\uFFFD\u0062\uFFFD\u0063\uFFFD\uFFFD\u0064",</p>
</li>
<li>
<pre><code> "\x61\xF1\x80\x80\xE1\x80\xC2\x62\x80\x63\x80\xBF\x64".fix_invalid)
</code></pre>
</li>
<li>
<li>
<p>assert_equal("abcdefghijklmnopqrstuvwxyz\u0061\uFFFD\uFFFD\uFFFD\u0062\uFFFD\u0063\uFFFD\uFFFD\u0064",</p>
</li>
<li>
<pre><code> "abcdefghijklmnopqrstuvwxyz\x61\xF1\x80\x80\xE1\x80\xC2\x62\x80\x63\x80\xBF\x64".fix_invalid)
</code></pre>
</li>
<li>
<li>
<p>assert_equal("\uFFFD\u3042".encode("UTF-16BE"),</p>
</li>
<li>
<pre><code> "\xD8\x00\x30\x42".force_encoding(Encoding::UTF_16BE).
</code></pre>
</li>
<li>
<pre><code> fix_invalid)
</code></pre>
</li>
<li>
<p>assert_equal("\uFFFD\u3042".encode("UTF-16LE"),</p>
</li>
<li>
<pre><code> "\x00\xD8\x42\x30".force_encoding(Encoding::UTF_16LE).
</code></pre>
</li>
<li>
<pre><code> fix_invalid)
</code></pre>
</li>
<li>
<p>end<br>
end</p>
</li>
</ul>
<p>class TestString2 < TestString<br>
=end</p>
Ruby master - Feature #6118 (Feedback): Hash#keys_of(values), returns related keys of given values
https://redmine.ruby-lang.org/issues/6118
2012-03-06T17:55:07Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Hash#key(value) の複数版がほしいです。</p>
<p>{a: 1, b: 2, c: 3, d: 1}.key(1)<br>
=> :a<br>
というメソッドはあるのですが、<br>
{a: 1, b: 2, c: 3, d: 1}.keys_of(1)<br>
=> [:a, :d]<br>
というメソッドは現状ありません。</p>
<p>Ruby での実装例は以下のような感じになります。<br>
どうでしょうか。</p>
<p>class Hash<br>
def keys_of(*a)<br>
each_with_object([]) {|(k, v), r| r << k if a.include? v}<br>
end<br>
end</p>
Ruby master - Feature #5959 (Rejected): Addrinfo#inspectname
https://redmine.ruby-lang.org/issues/5959
2012-02-02T17:16:51Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Addrinfo の導入によって、Socket.getaddrinfo での正引き結果がオブジェクト化され、<br>
元のホスト名を inspect で見れるようになってとても便利になったわけですが、<br>
現在この元のホスト名をとりだす API がありません。</p>
<p>ホスト名とIP アドレスをセットで扱うのに、つまり解決済みの名前とアドレスの組を扱うのに Addrinfo って便利なので、<br>
ホスト名を取り出す API が欲しいです。<br>
現在の C での名前、inspectname か hostname あたりでどうでしょう。</p>
Ruby master - Feature #5861 (Rejected): String#version_compare
https://redmine.ruby-lang.org/issues/5861
2012-01-08T07:54:53Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>バージョン番号っぽい文字列を比較するメソッド、String#version_compare を追加しませんか。<br>
"2.6.18".version_compare("2.6.3") #=> 1<br>
などと使います。</p>
<p>詳細な仕様は Gauche の gauche.version モジュールの version-compare 関数の仕様を丸パクリするのが良いと思います。<br>
提案している名前も Gauche そのままです。<br>
<a href="http://practical-scheme.net/gauche/man/gauche-refj_103.html" class="external">http://practical-scheme.net/gauche/man/gauche-refj_103.html</a></p>
<p>それなりにユースケースはある…というか今まさにテストを書いていて、<br>
Linux カーネルのバージョン番号を欲しくなったのですが、いかがでしょうか。</p>
Ruby master - Feature #5820 (Closed): Merge Onigmo to Ruby 2.0
https://redmine.ruby-lang.org/issues/5820
2011-12-29T02:42:18Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Ruby 1.9 では正規表現エンジンや M17N の基盤として Oniguruma を用いています。<br>
これを 2.0 では Oniguruma の改造版である、k-takata さんの Onigmo に置き換えようという話です。<br>
<a href="https://github.com/k-takata/Onigmo/tree/tmp/ruby-2.0.x" class="external">https://github.com/k-takata/Onigmo/tree/tmp/ruby-2.0.x</a></p>
<p>この取り込みによる影響は以下の通りです。</p>
<ul>
<li>100%互換 (既存のテストが全て無修正で通る)</li>
<li>いくつかの新機能 <a href="/issues/5208">[ruby-dev:44410]</a>
<ul>
<li>正規表現<br>
* \K, \R, \X, (?(cond)yes|no), \g<0>, \g<+n>, (?au)<br>
* Perl 5.10互換の名前参照(←Rubyには不要でしょう。)
<ul>
<li>Shift_JIS, EUC-JPで、全角アルファベットなどの大文字小文字同一視検索に対応。</li>
<li>Shift_JIS, EUC-JPで、\p{Han}, \p{Latin}, \p{Greek}, \p{Cyrillic} に対応。</li>
<li>最適化
<ul>
<li>暗黙のアンカーによる最適化を実装。</li>
<li>
<a href="http://redmine.ruby-lang.org/issues/3568" class="external">http://redmine.ruby-lang.org/issues/3568</a> で無効化された最適化を再度有効化。</li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
</ul>
<p>現状は POSIX 文字クラスに非互換があり、それが解決されればマージ可能と認識しています。</p>
Backport193 - Backport #5757 (Closed): main threadがreadやselectで待っていると、^C でなかなか死なない
https://redmine.ruby-lang.org/issues/5757
2011-12-13T16:35:41Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>FreeBSD 9 にて、 ./ruby と起動して ^C を投げてもなかなか死にません。<br>
./miniruby でも -e'$stdin.read' でも同じです。</p>
<p>仕組みとしては、main thread が read や select で待つ場合、最近は blocking region で<br>
unblock.func に ubf_select を設定するわけですが、この時にシグナルが来ると、</p>
<ol>
<li>どこかのスレッドの sighandler が呼ばれて、rb_thread_wakeup_timer_thread() が呼ばれる</li>
<li>タイマースレッドが起きて、thread_timer() -> timer_thread_function() -> rb_threadptr_check_signal() -> rb_threadptr_interrupt() -> (th->unblock.func)(th->unblock.arg) -> ubf_select() -> rb_thread_wakeup_timer_thread() が呼ばれる</li>
<li>タイマースレッドが起きて、thread_timer() -> timer_thread_function() -> rb_threadptr_check_signal() -> rb_threadptr_interrupt() -> (th->unblock.func)(th->unblock.arg) -> ubf_select() -> rb_thread_wakeup_timer_thread() が呼ばれる</li>
<li>タイマースレッドが起きて、thread_timer() -> timer_thread_function() -> rb_threadptr_check_signal() -> rb_threadptr_interrupt() -> (th->unblock.func)(th->unblock.arg) -> ubf_select() -> rb_thread_wakeup_timer_thread() が呼ばれる<br>
...</li>
</ol>
<p>対策はいくつかあり得ると思うのですが、例えば、ubf_select() から rb_thread_wakeup_timer_thread() を呼ばないようにするとか</p>
Ruby master - Feature #5282 (Closed): test-all 結果の順序
https://redmine.ruby-lang.org/issues/5282
2011-09-06T20:21:56Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>現在 Failure と Error と Skip が混ざって結果が出てくるので、ソートして欲しい。</p>
<p>具体的な順番は、一瞬 Skip を最後にしたくなるのだが、通常 EF は数個でかつ末尾から見る訳なので、<br>
Skip Failure Error の順がよいのではなかろうか。</p>
<p>ついでに、一番最後に ruby -v を出力してくれるとうれしいです。</p>
Backport193 - Backport #5276 (Closed): 4294967295.8.round is 4294967295 on 32bit
https://redmine.ruby-lang.org/issues/5276
2011-09-05T18:01:46Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>ruby -e'p 4294967295.8.round' must be 4294967296 but 4294967295 on 32bit environment.</p>
Ruby master - Feature #5180 (Closed): net/http の接続時に用いる IP アドレスの指定
https://redmine.ruby-lang.org/issues/5180
2011-08-10T11:45:57Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>通常 net/http を使う時は、Net::HTTP.start("ruby-lang.org") などとホスト名を使います。<br>
で、Socket がホスト名から IP アドレスを引いて、コネクションが張られます。<br>
普通の人はこれで足りるわけですが、ふつうな人はしばしば DNS で引けない IP アドレスに接続したくなります。<br>
例えば、ホスト名は "ruby-lang.org" としたいが、IP アドレスは 127.0.0.1 とか。</p>
<p>以下のパッチをあてると、<br>
Net::HTTP.start("ruby-lang.org", ipaddr: '127.0.0.1')<br>
などとできるようになります。</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/lib/net/http.rb b/lib/net/http.rb
index 7b9ec4f..6d034e0 100644
</span><span class="gd">--- a/lib/net/http.rb
</span><span class="gi">+++ b/lib/net/http.rb
</span><span class="p">@@ -524,7 +524,7 @@</span> module Net #:nodoc:
# _opt_ :: optional hash
#
# _opt_ sets following values by its accessor.
<span class="gd">- # The keys are ca_file, ca_path, cert, cert_store, ciphers,
</span><span class="gi">+ # The keys are ipaddr, ca_file, ca_path, cert, cert_store, ciphers,
</span> # close_on_empty_response, key, open_timeout, read_timeout, ssl_timeout,
# ssl_version, use_ssl, verify_callback, verify_depth and verify_mode.
# If you set :use_ssl as true, you can use https and default value of
<span class="p">@@ -542,6 +542,7 @@</span> module Net #:nodoc:
port, p_addr, p_port, p_user, p_pass = *arg
port = https_default_port if !port && opt && opt[:use_ssl]
http = new(address, port, p_addr, p_port, p_user, p_pass)
<span class="gi">+ http.ipaddr = opt[:ipaddr] if opt[:ipaddr]
</span>
if opt
if opt[:use_ssl]
<span class="p">@@ -575,6 +576,7 @@</span> module Net #:nodoc:
def initialize(address, port = nil)
@address = address
@port = (port || HTTP.default_port)
<span class="gi">+ @ipaddr = nil
</span> @curr_http_version = HTTPVersion
@no_keepalive_server = false
@close_on_empty_response = false
<span class="p">@@ -620,6 +622,17 @@</span> module Net #:nodoc:
# The port number to connect to.
attr_reader :port
<span class="gi">+ # The IP address to connect to/used to connect to
+ def ipaddr
+ started? ? @socket.io.peeraddr[3] : @ipaddr
+ end
+
+ # Set the IP address to connect to
+ def ipaddr=(addr)
+ raise IOError, "ipaddr value changed, but session already started" if started?
+ @ipaddr = addr
+ end
+
</span> # Number of seconds to wait for the connection to open. Any number
# may be used, including Floats for fractional seconds. If the HTTP
# object cannot open a connection in this many seconds, it raises a
<span class="p">@@ -945,7 +958,7 @@</span> module Net #:nodoc:
# without proxy
def conn_address
<span class="gd">- address()
</span><span class="gi">+ @ipaddr || address()
</span> end
def conn_port
</code></pre>
Ruby master - Feature #5153 (Closed): Remove rb_add_suffix
https://redmine.ruby-lang.org/issues/5153
2011-08-03T12:11:46Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>rb_add_suffix in util.c is GPL and obsoleted code, so I want to remove it.</p>
<p>rb_add_suffix is drived from Perl's win32.c file and GPL/Artistic License.<br>
The code is used when ruby runs with -i (inplace option).<br>
Practically it works only when the renamed file can't create.<br>
(the validation is also a function of the code, but it is not essential)</p>
<p>But this behavior is Windows specific.<br>
On other environment, ruby simply skip the file with a warning "Can't rename %s to %s: %s, skipping file".<br>
I think Windows should follow this.</p>
Ruby master - Feature #5142 (Rejected): Remove ruby-mode.el from ruby's repo
https://redmine.ruby-lang.org/issues/5142
2011-08-02T10:46:33Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>ruby-mode.el は Emacs に添付されるようになりましたが、<br>
今も Ruby のリポジトリ内に misc/ruby-mode.el として存在し続けています。<br>
マスタが Ruby と Emacs の2つあるのも良くないですし、放置されているチケットも多いので、<br>
Ruby のリポジトリからは削除してしまいませんか。</p>
<p>なお、ruby-mode.el 以外の *.el についてはこのチケットの管轄外とします。</p>
Backport193 - Backport #5130 (Closed): Thread.pass sticks on OpenBSD
https://redmine.ruby-lang.org/issues/5130
2011-08-01T15:51:28Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
On OpenBSD 4.9, following script will stick.</p>
<p>./miniruby -ve'Thread.new{Thread.pass}'<br>
=end</p>
Ruby master - Feature #5097 (Closed): Supported platforms of Ruby 1.9.3
https://redmine.ruby-lang.org/issues/5097
2011-07-26T11:52:57Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Let's decide the supported platforms.</p>
<p>== Background</p>
<p><a href="http://redmine.ruby-lang.org/projects/ruby-19/wiki/SupportedPlatforms" class="external">http://redmine.ruby-lang.org/projects/ruby-19/wiki/SupportedPlatforms</a></p>
<p>== Process</p>
<p>If you want to support a platform, please declare.<br>
But when a platform dependent bug is reported, it will be assigned to you.</p>
<p>== Current Maintainer</p>
<p>mswin32, mswin64 (Microsoft Windows):<br>
NAKAMURA Usaku (usa)<br>
mingw32 (Minimalist GNU for Windows):<br>
Nobuyoshi Nakada (nobu)<br>
IA-64 (Debian GNU/Linux):<br>
TAKANO Mitsuhiro (takano32)<br>
Symbian OS:<br>
Alexander Zavorine (azov)<br>
AIX:<br>
Yutaka Kanemoto (kanemoto)<br>
FreeBSD:<br>
Akinori MUSHA (knu)<br>
Solaris:<br>
Naohisa Goto<br>
RHEL, CentOS<br>
KOSAKI Motohiro</p>
<p>Platforms which doesn't have a maintainer are following:</p>
<ul>
<li>Debian</li>
<li>Ubuntu</li>
<li>Mac OS X (LLVM related issues)</li>
<li>cygwin (don't work)</li>
<li>NetBSD (works)</li>
<li>OpenBSD (it may not work)</li>
<li>DragonFlyBSD (don't work)</li>
</ul>
Backport192 - Backport #5075 (Rejected): invalid *fdp in Mac OS X and FreeBSD over recvmsg with S...
https://redmine.ruby-lang.org/issues/5075
2011-07-22T17:51:06Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>Mac OS X と FreeBSD にて、存在しない fd を close してしまう問題について、<br>
現在 r32598 で応急処置が施されていますが、根本的な原因について、<br>
sys/kern/uipc_socket.c を見るに、<br>
<a href="http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/uipc_socket.c?rev=1.340.2.6.2.1;content-type=text%2Fplain;only_with_tag=RELENG_8_2_0_RELEASE" class="external">http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/uipc_socket.c?rev=1.340.2.6.2.1;content-type=text%2Fplain;only_with_tag=RELENG_8_2_0_RELEASE</a></p>
<pre><code> * Process one or more MT_CONTROL mbufs present before any data mbufs
* in the first mbuf chain on the socket buffer. If MSG_PEEK, we
* just copy the data; if !MSG_PEEK, we call into the protocol to
* perform externalization (or freeing if controlp == NULL).
</code></pre>
<p>とあるので、recvmsg に MSG_PEEK を与えた場合は invalid なものが返ってくると思うのですが。</p>
<p>ちなみに、以下のような printf パッチをあてて走らせると、discard_cmsg() に来たものは全て invalid になっています。</p>
<p>diff --git a/ext/socket/ancdata.c b/ext/socket/ancdata.c<br>
index 61e0576..ad44fb4 100644<br>
--- a/ext/socket/ancdata.c<br>
+++ b/ext/socket/ancdata.c<br>
@@ -1379,6 +1379,7 @@ rb_recvmsg(int fd, struct msghdr *msg, int flags)<br>
static void<br>
discard_cmsg(struct cmsghdr *cmh, char *msg_end)<br>
{</p>
<ul>
<li>fprintf(stderr, "discard_cmsg-begin\n");<br>
if (cmh->cmsg_level == SOL_SOCKET && cmh->cmsg_type == SCM_RIGHTS) {<br>
int *fdp = (int *)CMSG_DATA(cmh);<br>
int *end = (int *)((char *)cmh + cmh->cmsg_len);<br>
@@ -1391,12 +1392,18 @@ discard_cmsg(struct cmsghdr *cmh, char *msg_end)<br>
*/<br>
struct stat buf;<br>
if (fstat(*fdp, &buf) == 0) {</li>
<li>
<pre><code> fprintf(stderr, "fdp: %d is valid (%p %p %p)\n", *fdp,fdp,end,msg_end);
rb_update_max_fd(*fdp);
close(*fdp);
}
</code></pre>
</li>
<li>
<pre><code> else {
</code></pre>
</li>
<li>
<pre><code> fprintf(stderr, "fdp: %d is invalid (%p %p %p)\n", *fdp,fdp,end,msg_end);
</code></pre>
</li>
<li>
<pre><code> rb_backtrace();
</code></pre>
</li>
<li>
<pre><code> }
fdp++;
}
</code></pre>
}</li>
<li>fprintf(stderr, "discard_cmsg-end\n");<br>
}<br>
#endif</li>
</ul>
<p>@@ -1432,6 +1439,7 @@ make_io_for_unix_rights(VALUE ctl, struct cmsghdr *cmh, char *msg_end)<br>
(char *)fdp + sizeof(int) <= msg_end) {<br>
int fd = *fdp;<br>
struct stat stbuf;</p>
<ul>
<li>fprintf(stderr,"makeiounixr: %d (%p %p %p)\n", *fdp,fdp,end,msg_end);<br>
VALUE io;<br>
if (fstat(fd, &stbuf) == -1)<br>
rb_raise(rb_eSocket, "invalid fd in SCM_RIGHTS");</li>
</ul>
Ruby master - Feature #4921 (Rejected): Remove intern.h
https://redmine.ruby-lang.org/issues/4921
2011-06-23T08:15:07Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>まず、intern.h って何者なんでしょう。<br>
推測としては internal の意だと思うんですが、その場合 include/ruby にいるのは<br>
よろしくないですし、一方で公開 API っぽいのが名実ともに多い気がします。</p>
<p>思うに 1.9 において、intern.h の中身は include/ruby/ruby.h か、<br>
最近新設された internal.h のどちらかにあるべきで、include/ruby/intern.h は<br>
もう必要ないのではないでしょうか。</p>
Ruby master - Feature #4595 (Closed): TkPhotoImage documentation
https://redmine.ruby-lang.org/issues/4595
2011-04-22T12:12:22Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
以下のような pull request が来ています。</p>
<p>I have documented some method from TkPhotoImage, based on the original Tcl/Tk docs.</p>
<p><a href="https://github.com/ruby/ruby/pull/9" class="external">https://github.com/ruby/ruby/pull/9</a><br>
=end</p>
Ruby master - Feature #4529 (Rejected): date_core と long 型
https://redmine.ruby-lang.org/issues/4529
2011-03-26T06:13:43Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
おそらく既にご存じの通り、最近 date_core も含めて ext 中の shorten-64-to-32 を直しています。<br>
それに際して date_core をいじっていて気づいたのですが、ユリウス日や年を long で保持したり int で保持したりしています。</p>
<p>int と long が混在していると、long から int へのキャスト時にオーバーフローが起こりえますし、<br>
そもそも long は</p>
<ul>
<li>環境によってサイズが違う (int も 理屈の上では違うはずだが、CRuby では 32bit しか確認してない)</li>
<li>long は mswin64 でだけポインタとサイズが違うので、Unix な人には気づきづらいバグの温床になる<br>
というような懸念があるので、特に理由が無いのであれば、int にするか、int64_t にするかした方がよいかと思います。<br>
(CRuby は C90 だが、int64_t を configure で定義している)</li>
</ul>
<p>例えば int に統一する場合のパッチは以下の通りです</p>
<p>=end</p>
Ruby master - Feature #4180 (Closed): Add Zlib.deflate / Zlib.inflate
https://redmine.ruby-lang.org/issues/4180
2010-12-21T10:34:56Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
Zlib::Deflate.deflate ってムダに長いと思いんです。<br>
Rails 的な言葉で言うと、DRY じゃないと言いますか。<br>
Zlib.deflate でいいんじゃないかと。</p>
<p>同様に、Zlib::Inflate.inflate も Zlib.inflate でいいのではないでしょうか。<br>
=end</p>
Backport187 - Backport #4171 (Closed): Warn Array#choice
https://redmine.ruby-lang.org/issues/4171
2010-12-19T18:26:40Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
Array#choice は Ruby 1.8.7 で入ったメソッドですが、1.9 ではなくなっているので、<br>
ruby_1_8 だけでなく、ruby_1_8_7 パッチリリースでも warning を出すようにしませんか。<br>
=end</p>
Ruby master - Feature #4142 (Closed): multipart/form-data for net/http
https://redmine.ruby-lang.org/issues/4142
2010-12-10T09:17:39Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
multipart/form-data 対応を net/http に入れませんか。<br>
追加される API は Net::HTTPRequest#set_form になります。</p>
<p>akr さんからは multipart/form-data 用のデータを出力する API 案も示唆されたのですが、<br>
chunked encoding を考慮に入れるとうまくまとまらなかったので見送っています。</p>
<p>diff --git a/lib/net/http.rb b/lib/net/http.rb<br>
index 4d475b1..2751f77 100644<br>
--- a/lib/net/http.rb<br>
+++ b/lib/net/http.rb<br>
@@ -22,6 +22,7 @@<br>
require 'net/protocol'<br>
autoload :OpenSSL, 'openssl'<br>
require 'uri'<br>
+autoload :SecureRandom, 'securerandom'</p>
<p>module Net #:nodoc:</p>
<p>@@ -1772,7 +1773,8 @@ module Net #:nodoc:<br>
alias content_type= set_content_type</p>
<pre><code> # Set header fields and a body from HTML form data.
</code></pre>
<ul>
<li>
<a name="params-should-be-a-Hash-containing-HTML-form-data"></a>
<h1 >+params+ should be a Hash containing HTML form data.<a href="#params-should-be-a-Hash-containing-HTML-form-data" class="wiki-anchor">¶</a></h1>
</li>
</ul>
<ul>
<li>
<a name="params-should-be-an-Array-of-Arrays-or"></a>
<h1 >+params+ should be an Array of Arrays or<a href="#params-should-be-an-Array-of-Arrays-or" class="wiki-anchor">¶</a></h1>
</li>
<li>
<a name="a-Hash-containing-HTML-form-data"></a>
<h1 >a Hash containing HTML form data.<a href="#a-Hash-containing-HTML-form-data" class="wiki-anchor">¶</a></h1>
<a name="Optional-argument-sep-means-data-record-separator"></a>
<h1 >Optional argument +sep+ means data record separator.<a href="#Optional-argument-sep-means-data-record-separator" class="wiki-anchor">¶</a></h1>
<h1></h1>
<a name="Values-are-URL-encoded-as-necessary-and-the-content-type-is-set-to"></a>
<h1 >Values are URL encoded as necessary and the content-type is set to<a href="#Values-are-URL-encoded-as-necessary-and-the-content-type-is-set-to" class="wiki-anchor">¶</a></h1>
</li>
</ul>
<p>@@ -1792,6 +1794,48 @@ module Net #:nodoc:</p>
<pre><code> alias form_data= set_form_data
</code></pre>
<ul>
<li>
<a name="Set-a-HTML-form-data-set"></a>
<h1 >Set a HTML form data set.<a href="#Set-a-HTML-form-data-set" class="wiki-anchor">¶</a></h1>
</li>
<li>
<a name="params-is-the-form-data-set-it-is-an-Array-of-Arrays-or-a-Hash"></a>
<h1 >+params+ is the form data set; it is an Array of Arrays or a Hash<a href="#params-is-the-form-data-set-it-is-an-Array-of-Arrays-or-a-Hash" class="wiki-anchor">¶</a></h1>
</li>
<li>
<a name="enctype-is-the-type-to-encode-the-form-data-set"></a>
<h1 >+enctype is the type to encode the form data set.<a href="#enctype-is-the-type-to-encode-the-form-data-set" class="wiki-anchor">¶</a></h1>
</li>
<li>
<a name="It-is-applicationx-www-form-urlencoded-or-multipartform-data"></a>
<h1 >It is application/x-www-form-urlencoded or multipart/form-data.<a href="#It-is-applicationx-www-form-urlencoded-or-multipartform-data" class="wiki-anchor">¶</a></h1>
</li>
<li>
<a name="formpot-is-an-optional-hash-to-specify-the-detail"></a>
<h1 >+formpot+ is an optional hash to specify the detail.<a href="#formpot-is-an-optional-hash-to-specify-the-detail" class="wiki-anchor">¶</a></h1>
</li>
<li>
<h1></h1>
</li>
<li>
<a name="boundary-the-boundary-of-the-multipart-message"></a>
<h1 >boundary:: the boundary of the multipart message<a href="#boundary-the-boundary-of-the-multipart-message" class="wiki-anchor">¶</a></h1>
</li>
<li>
<a name="charset-the-charset-of-the-message-All-names-and-the-values-of"></a>
<h1 >charset:: the charset of the message. All names and the values of<a href="#charset-the-charset-of-the-message-All-names-and-the-values-of" class="wiki-anchor">¶</a></h1>
</li>
<li>
<a name="non-file-fields-are-encoded-as-the-charset"></a>
<h1 >non-file fields are encoded as the charset.<a href="#non-file-fields-are-encoded-as-the-charset" class="wiki-anchor">¶</a></h1>
</li>
<li>
<h1></h1>
</li>
<li>
<a name="Each-item-of-params-is-an-array-and-contains-following-items"></a>
<h1 >Each item of params is an array and contains following items:<a href="#Each-item-of-params-is-an-array-and-contains-following-items" class="wiki-anchor">¶</a></h1>
</li>
<li>
<a name="name-the-name-of-the-field"></a>
<h1 >+name+:: the name of the field<a href="#name-the-name-of-the-field" class="wiki-anchor">¶</a></h1>
</li>
<li>
<a name="value-the-value-of-the-field-it-should-be-a-String-or-a-File"></a>
<h1 >+value+:: the value of the field, it should be a String or a File<a href="#value-the-value-of-the-field-it-should-be-a-String-or-a-File" class="wiki-anchor">¶</a></h1>
</li>
<li>
<a name="opt-an-optional-hash-to-specify-additional-information"></a>
<h1 >+opt+:: an optional hash to specify additional information<a href="#opt-an-optional-hash-to-specify-additional-information" class="wiki-anchor">¶</a></h1>
</li>
<li>
<h1></h1>
</li>
<li>
<a name="Each-item-is-a-file-field-or-a-normal-field"></a>
<h1 >Each item is a file field or a normal field.<a href="#Each-item-is-a-file-field-or-a-normal-field" class="wiki-anchor">¶</a></h1>
</li>
<li>
<a name="If-value-is-a-File-object-or-the-opt-have-a-filename-key"></a>
<h1 >If +value+ is a File object or the +opt+ have a filename key,<a href="#If-value-is-a-File-object-or-the-opt-have-a-filename-key" class="wiki-anchor">¶</a></h1>
</li>
<li>
<a name="the-item-is-treated-as-a-file-field"></a>
<h1 >the item is treated as a file field.<a href="#the-item-is-treated-as-a-file-field" class="wiki-anchor">¶</a></h1>
</li>
<li>
<h1></h1>
</li>
<li>
<a name="If-Transfer-Encoding-is-set-as-chunked-this-send-the-request-in"></a>
<h1 >If Transfer-Encoding is set as chunked, this send the request in<a href="#If-Transfer-Encoding-is-set-as-chunked-this-send-the-request-in" class="wiki-anchor">¶</a></h1>
</li>
<li>
<a name="chunked-encoding-Because-chunked-encoding-is-HTTP11-feature"></a>
<h1 >chunked encoding. Because chunked encoding is HTTP/1.1 feature,<a href="#chunked-encoding-Because-chunked-encoding-is-HTTP11-feature" class="wiki-anchor">¶</a></h1>
</li>
<li>
<a name="you-must-confirm-the-server-to-support-HTTP11-before-sending-it"></a>
<h1 >you must confirm the server to support HTTP/1.1 before sending it.<a href="#you-must-confirm-the-server-to-support-HTTP11-before-sending-it" class="wiki-anchor">¶</a></h1>
</li>
<li>
<h1></h1>
</li>
<li>
<a name="Example"></a>
<h1 >Example:<a href="#Example" class="wiki-anchor">¶</a></h1>
</li>
<li>
<a name="httpset_formq-ruby-lang-en"></a>
<h1 >http.set_form([["q", "ruby"], ["lang", "en"]])<a href="#httpset_formq-ruby-lang-en" class="wiki-anchor">¶</a></h1>
</li>
<li>
<h1></h1>
</li>
<li>
<a name="See-also-RFC-2388-RFC-2616-HTML-401-and-HTML5"></a>
<h1 >See also RFC 2388, RFC 2616, HTML 4.01, and HTML5<a href="#See-also-RFC-2388-RFC-2616-HTML-401-and-HTML5" class="wiki-anchor">¶</a></h1>
</li>
<li>
<h1></h1>
</li>
<li>
<p>def set_form(params, enctype='application/x-www-form-urlencoded', formopt={})</p>
</li>
<li>
<pre><code> @body_data = params
</code></pre>
</li>
<li>
<pre><code> @body = nil
</code></pre>
</li>
<li>
<pre><code> @body_stream = nil
</code></pre>
</li>
<li>
<pre><code> @form_option = formopt
</code></pre>
</li>
<li>
<pre><code> case enctype
</code></pre>
</li>
<li>
<pre><code> when /\Aapplication\/x-www-form-urlencoded\z/i,
</code></pre>
</li>
<li>
<pre><code> /\Amultipart\/form-data\z/i
</code></pre>
</li>
<li>
<pre><code> self.content_type = enctype
</code></pre>
</li>
<li>
<pre><code> else
</code></pre>
</li>
<li>
<pre><code> raise ArgumentError, "invalid enctype: #{enctype}"
</code></pre>
</li>
<li>
<pre><code> end
</code></pre>
</li>
<li>
<p>end</p>
</li>
<li>
<a name="Set-the-Authorization-header-for-Basic-authorization"></a>
<h1 >Set the Authorization: header for "Basic" authorization.<a href="#Set-the-Authorization-header-for-Basic-authorization" class="wiki-anchor">¶</a></h1>
<p>def basic_auth(account, password)<br>
@header['authorization'] = [basic_encode(account, password)]<br>
@@ -1849,6 +1893,7 @@ module Net #:nodoc:<br>
self['User-Agent'] ||= 'Ruby'<br>
@body = nil<br>
@body_stream = nil</p>
</li>
<li>
<pre><code> @body_data = nil
</code></pre>
<p>end</p>
<p>attr_reader :method<br>
@@ -1876,6 +1921,7 @@ module Net #:nodoc:<br>
def body=(str)<br>
@body = str<br>
@body_stream = nil</p>
</li>
<li>
<pre><code> @body_data = nil
str
</code></pre>
<p>end</p>
</li>
</ul>
<p>@@ -1884,6 +1930,7 @@ module Net #:nodoc:<br>
def body_stream=(input)<br>
@body = nil<br>
@body_stream = input</p>
<ul>
<li>
<pre><code> @body_data = nil
input
</code></pre>
end</li>
</ul>
<p>@@ -1901,6 +1948,8 @@ module Net #:nodoc:<br>
send_request_with_body sock, ver, path, @body<br>
elsif @body_stream<br>
send_request_with_body_stream sock, ver, path, @body_stream</p>
<ul>
<li>
<pre><code> elsif @body_data
</code></pre>
</li>
<li>
<pre><code> send_request_with_body_data sock, ver, path, @body_data
else
write_header sock, ver, path
end
</code></pre>
</li>
</ul>
<p>@@ -1935,6 +1984,92 @@ module Net #:nodoc:<br>
end<br>
end</p>
<ul>
<li>
<p>def send_request_with_body_data(sock, ver, path, params)</p>
</li>
<li>
<pre><code> if /\Amultipart\/form-data\z/i !~ self.content_type
</code></pre>
</li>
<li>
<pre><code> self.content_type = 'application/x-www-form-urlencoded'
</code></pre>
</li>
<li>
<pre><code> return send_request_with_body(sock, ver, path, URI.encode_www_form(params))
</code></pre>
</li>
<li>
<pre><code> end
</code></pre>
</li>
<li>
<li>
<pre><code> opt = @form_option.dup
</code></pre>
</li>
<li>
<pre><code> opt[:boundary] ||= SecureRandom.urlsafe_base64(40)
</code></pre>
</li>
<li>
<pre><code> self.set_content_type(self.content_type, boundary: opt[:boundary])
</code></pre>
</li>
<li>
<pre><code> if chunked?
</code></pre>
</li>
<li>
<pre><code> write_header sock, ver, path
</code></pre>
</li>
<li>
<pre><code> encode_multipart_form_data(sock, params, opt)
</code></pre>
</li>
<li>
<pre><code> else
</code></pre>
</li>
<li>
<pre><code> require 'tempfile'
</code></pre>
</li>
<li>
<pre><code> file = Tempfile.new('multipart')
</code></pre>
</li>
<li>
<pre><code> encode_multipart_form_data(file, params, opt)
</code></pre>
</li>
<li>
<pre><code> file.rewind
</code></pre>
</li>
<li>
<pre><code> self.content_length = file.size
</code></pre>
</li>
<li>
<pre><code> write_header sock, ver, path
</code></pre>
</li>
<li>
<pre><code> IO.copy_stream(file, sock)
</code></pre>
</li>
<li>
<pre><code> end
</code></pre>
</li>
<li>
<p>end</p>
</li>
<li>
<li>
<p>def encode_multipart_form_data(out, params, opt)</p>
</li>
<li>
<pre><code> charset = opt[:charset]
</code></pre>
</li>
<li>
<pre><code> boundary = opt[:boundary]
</code></pre>
</li>
<li>
<pre><code> boundary ||= SecureRandom.urlsafe_base64(40)
</code></pre>
</li>
<li>
<pre><code> chunked_p = chunked?
</code></pre>
</li>
<li>
<li>
<pre><code> buf = ''
</code></pre>
</li>
<li>
<pre><code> params.each do |key, value, h={}|
</code></pre>
</li>
<li>
<pre><code> key = quote_string(key, charset)
</code></pre>
</li>
<li>
<pre><code> filename =
</code></pre>
</li>
<li>
<pre><code> h.key?(:filename) ? h[:filename] :
</code></pre>
</li>
<li>
<pre><code> value.respond_to?(:to_path) ? File.basename(value.to_path) :
</code></pre>
</li>
<li>
<pre><code> nil
</code></pre>
</li>
<li>
<li>
<pre><code> buf << "--#{boundary}\r\n"
</code></pre>
</li>
<li>
<pre><code> if filename
</code></pre>
</li>
<li>
<pre><code> filename = quote_string(filename, charset)
</code></pre>
</li>
<li>
<pre><code> type = h[:content_type] || 'application/octet-stream'
</code></pre>
</li>
<li>
<pre><code> buf << "Content-Disposition: form-data; " \
</code></pre>
</li>
<li>
<pre><code> "name=\"#{key}\"; filename=\"#{filename}\"\r\n" \
</code></pre>
</li>
<li>
<pre><code> "Content-Type: #{type}\r\n\r\n"
</code></pre>
</li>
<li>
<pre><code> if !out.respond_to?(:write) || !value.respond_to?(:read)
</code></pre>
</li>
<li>
<pre><code> # if +out+ is not an IO or +value+ is not an IO
</code></pre>
</li>
<li>
<pre><code> buf << (value.respond_to?(:read) ? value.read : value)
</code></pre>
</li>
<li>
<pre><code> elsif value.respond_to?(:size) && chunked_p
</code></pre>
</li>
<li>
<pre><code> # if +out+ is an IO and +value+ is a File, use IO.copy_stream
</code></pre>
</li>
<li>
<pre><code> flush_buffer(out, buf, chunked_p)
</code></pre>
</li>
<li>
<pre><code> out << "%x\r\n" % value.size if chunked_p
</code></pre>
</li>
<li>
<pre><code> IO.copy_stream(value, out)
</code></pre>
</li>
<li>
<pre><code> out << "\r\n" if chunked_p
</code></pre>
</li>
<li>
<pre><code> else
</code></pre>
</li>
<li>
<pre><code> # +out+ is an IO, and +value+ is not a File but an IO
</code></pre>
</li>
<li>
<pre><code> flush_buffer(out, buf, chunked_p)
</code></pre>
</li>
<li>
<pre><code> 1 while flush_buffer(out, value.read(4096), chunked_p)
</code></pre>
</li>
<li>
<pre><code> end
</code></pre>
</li>
<li>
<pre><code> else
</code></pre>
</li>
<li>
<pre><code> # non-file field:
</code></pre>
</li>
<li>
<pre><code> # HTML5 says, "The parts of the generated multipart/form-data
</code></pre>
</li>
<li>
<pre><code> # resource that correspond to non-file fields must not have a
</code></pre>
</li>
<li>
<pre><code> # Content-Type header specified."
</code></pre>
</li>
<li>
<pre><code> buf << "Content-Disposition: form-data; name=\"#{key}\"\r\n\r\n"
</code></pre>
</li>
<li>
<pre><code> buf << (value.respond_to?(:read) ? value.read : value)
</code></pre>
</li>
<li>
<pre><code> end
</code></pre>
</li>
<li>
<pre><code> buf << "\r\n"
</code></pre>
</li>
<li>
<pre><code> end
</code></pre>
</li>
<li>
<pre><code> buf << "--#{boundary}--\r\n"
</code></pre>
</li>
<li>
<pre><code> flush_buffer(out, buf, chunked_p)
</code></pre>
</li>
<li>
<pre><code> out << "0\r\n\r\n" if chunked_p
</code></pre>
</li>
<li>
<p>end</p>
</li>
<li>
<li>
<p>def quote_string(str, charset)</p>
</li>
<li>
<pre><code> str = str.encode(charset, fallback:->(c){'&#%d;'%c.encode("UTF-8").ord}) if charset
</code></pre>
</li>
<li>
<pre><code> str = str.gsub(/[\\"]/, '\\\\\&')
</code></pre>
</li>
<li>
<p>end</p>
</li>
<li>
<li>
<p>def flush_buffer(out, buf, chunked_p)</p>
</li>
<li>
<pre><code> return unless buf
</code></pre>
</li>
<li>
<pre><code> out << "%x\r\n"%buf.bytesize if chunked_p
</code></pre>
</li>
<li>
<pre><code> out << buf
</code></pre>
</li>
<li>
<pre><code> out << "\r\n" if chunked_p
</code></pre>
</li>
<li>
<pre><code> buf.clear
</code></pre>
</li>
<li>
<p>end</p>
</li>
<li>
<p>def supply_default_content_type<br>
return if content_type()<br>
warn 'net/http: warning: Content-Type did not set; using application/x-www-form-urlencoded' if $VERBOSE<br>
diff --git a/lib/net/protocol.rb b/lib/net/protocol.rb<br>
index 2a6cfb4..a3ffa71 100644<br>
--- a/lib/net/protocol.rb<br>
+++ b/lib/net/protocol.rb<br>
@@ -168,6 +168,8 @@ module Net # :nodoc:<br>
}<br>
end</p>
</li>
<li>
<p>alias << write</p>
</li>
<li>
<p>def writeline(str)<br>
writing {<br>
write0 str + "\r\n"<br>
diff --git a/test/net/http/test_http.rb b/test/net/http/test_http.rb<br>
index 76280ad..12c03a4 100644<br>
--- a/test/net/http/test_http.rb<br>
+++ b/test/net/http/test_http.rb<br>
@@ -303,6 +303,102 @@ module TestNetHTTP_version_1_2_methods<br>
assert_equal data.size, res.body.size<br>
assert_equal data, res.body<br>
end</p>
</li>
<li>
<li>
<p>def test_set_form</p>
</li>
<li>
<p>require 'tempfile'</p>
</li>
<li>
<p>file = Tempfile.new('ruby-test')</p>
</li>
<li>
<p>file << "\u{30c7}\u{30fc}\u{30bf}"</p>
</li>
<li>
<p>data = [</p>
</li>
<li>
<pre><code> ['name', 'Gonbei Nanashi'],
</code></pre>
</li>
<li>
<pre><code> ['name', "\u{540d}\u{7121}\u{3057}\u{306e}\u{6a29}\u{5175}\u{885b}"],
</code></pre>
</li>
<li>
<pre><code> ['s"i\o', StringIO.new("\u{3042 3044 4e9c 925b}")],
</code></pre>
</li>
<li>
<pre><code> ["file", file, filename: "ruby-test"]
</code></pre>
</li>
<li>
<p>]</p>
</li>
<li>
<p>expected = <<"<strong>EOM</strong>".gsub(/\n/, "\r\n")<br>
+--<br>
+Content-Disposition: form-data; name="name"</p>
</li>
<li>
</ul>
<p>+Gonbei Nanashi<br>
+--<br>
+Content-Disposition: form-data; name="name"<br>
+<br>
+\xE5\x90\x8D\xE7\x84\xA1\xE3\x81\x97\xE3\x81\xAE\xE6\xA8\xA9\xE5\x85\xB5\xE8\xA1\x9B<br>
+--<br>
+Content-Disposition: form-data; name="s\"i\\o"<br>
+<br>
+\xE3\x81\x82\xE3\x81\x84\xE4\xBA\x9C\xE9\x89\x9B<br>
+--<br>
+Content-Disposition: form-data; name="file"; filename="ruby-test"<br>
+Content-Type: application/octet-stream<br>
+<br>
+\xE3\x83\x87\xE3\x83\xBC\xE3\x82\xBF<br>
+----<br>
+<strong>EOM</strong></p>
<ul>
<li>start {|http|</li>
<li>
<pre><code> _test_set_form_urlencoded(http, data.reject{|k,v|!v.is_a?(String)})
</code></pre>
</li>
<li>
<pre><code> _test_set_form_multipart(http, false, data, expected)
</code></pre>
</li>
<li>
<pre><code> _test_set_form_multipart(http, true, data, expected)
</code></pre>
</li>
<li>}</li>
<li>end</li>
<li>
<li>def _test_set_form_urlencoded(http, data)</li>
<li>req = Net::HTTP::Post.new('/')</li>
<li>req.set_form(data)</li>
<li>res = http.request req</li>
<li>assert_equal "name=Gonbei+Nanashi&name=%E5%90%8D%E7%84%A1%E3%81%97%E3%81%AE%E6%A8%A9%E5%85%B5%E8%A1%9B", res.body</li>
<li>end</li>
<li>
<li>def _test_set_form_multipart(http, chunked_p, data, expected)</li>
<li>data.each{|k,v|v.rewind rescue nil}</li>
<li>req = Net::HTTP::Post.new('/')</li>
<li>req.set_form(data, 'multipart/form-data')</li>
<li>req['Transfer-Encoding'] = 'chunked' if chunked_p</li>
<li>res = http.request req</li>
<li>body = res.body</li>
<li>assert_match(/\A--(?\S+)/, body)</li>
<li>/\A--(?\S+)/ =~ body</li>
<li>expected = expected.gsub(//, boundary)</li>
<li>assert_equal(expected, body)</li>
<li>end</li>
<li>
<li>def test_set_form_with_file</li>
<li>require 'tempfile'</li>
<li>file = Tempfile.new('ruby-test')</li>
<li>file << $test_net_http_data</li>
<li>filename = File.basename(file.to_path)</li>
<li>data = [['file', file]]</li>
<li>expected = <<"<strong>EOM</strong>".gsub(/\n/, "\r\n")<br>
+--<br>
+Content-Disposition: form-data; name="file"; filename=""<br>
+Content-Type: application/octet-stream</li>
<li>
</ul>
<p>+<br>
+----<br>
+<strong>EOM</strong></p>
<ul>
<li>expected.sub!(//, filename)</li>
<li>expected.sub!(//, $test_net_http_data)</li>
<li>start {|http|</li>
<li>
<pre><code> data.each{|k,v|v.rewind rescue nil}
</code></pre>
</li>
<li>
<pre><code> req = Net::HTTP::Post.new('/')
</code></pre>
</li>
<li>
<pre><code> req.set_form(data, 'multipart/form-data')
</code></pre>
</li>
<li>
<pre><code> res = http.request req
</code></pre>
</li>
<li>
<pre><code> body = res.body
</code></pre>
</li>
<li>
<pre><code> header, _ = body.split(/\r\n\r\n/, 2)
</code></pre>
</li>
<li>
<pre><code> assert_match(/\A--(?<boundary>\S+)/, body)
</code></pre>
</li>
<li>
<pre><code> /\A--(?<boundary>\S+)/ =~ body
</code></pre>
</li>
<li>
<pre><code> expected = expected.gsub(/<boundary>/, boundary)
</code></pre>
</li>
<li>
<pre><code> assert_match(/^--(?<boundary>\S+)\r\n/, header)
</code></pre>
</li>
<li>
<pre><code> assert_match(
</code></pre>
</li>
<li>
<pre><code> /^Content-Disposition: form-data; name="file"; filename="#{filename}"\r\n/,
</code></pre>
</li>
<li>
<pre><code> header)
</code></pre>
</li>
<li>
<pre><code> assert_equal(expected, body)
</code></pre>
</li>
<li>
<li>
<pre><code> data.each{|k,v|v.rewind rescue nil}
</code></pre>
</li>
<li>
<pre><code> req['Transfer-Encoding'] = 'chunked'
</code></pre>
</li>
<li>
<pre><code> res = http.request req
</code></pre>
</li>
<li>
<pre><code> #assert_equal(expected, res.body)
</code></pre>
</li>
<li>}</li>
<li>end<br>
end</li>
</ul>
<p>class TestNetHTTP_version_1_1 < Test::Unit::TestCase<br>
=end</p>
Ruby master - Feature #4089 (Closed): Add addr2line for C level backtrace
https://redmine.ruby-lang.org/issues/4089
2010-11-26T09:22:36Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
最近の Ruby は SEGV や BUG 時に Ruby level backtrace を出したり、<br>
取れるときは C level backtrace を出したりしています。</p>
<p>ところが、C level backtrace だけ見ても実のところあまり助けにならないことが多いので、<br>
ソースコードのファイルや行数も可能ならば出したいところです。</p>
<p>などと言っていたら浜地さんがパッチを作ってくれたので、これを取り込みませんか。<br>
glibc 環境 (つまり Linux) や、libexecinfo を導入している FreeBSD や NetBSD など (のELFなバイナリ) で動きます。</p>
<p>diff --git a/addr2line.c b/addr2line.c<br>
new file mode 100644<br>
index 0000000..da85f4d<br>
--- /dev/null<br>
+++ b/addr2line.c<br>
@@ -0,0 +1,534 @@<br>
+/**********************************************************************<br>
+</p>
<ul>
<li>addr2line.h -</li>
<li>
<li>$Author$</li>
<li>
<li>Copyright (C) 2010 Shinichiro Hamaji</li>
<li>
</ul>
<p>+*********************************************************************<em>/<br>
+<br>
+#include "addr2line.h"<br>
+<br>
+#include <stdio.h><br>
+<br>
+#ifdef <strong>ELF</strong><br>
+<br>
+#include <elf.h><br>
+#include <fcntl.h><br>
+#include <limits.h><br>
+#include <stdio.h><br>
+#include <stdlib.h><br>
+#include <string.h><br>
+#include <sys/mman.h><br>
+#include <sys/types.h><br>
+#include <sys/stat.h><br>
+#include <unistd.h><br>
+<br>
+#ifdef HAVE_DL_ITERATE_PHDR<br>
+# ifndef _GNU_SOURCE<br>
+# define _GNU_SOURCE<br>
+# endif<br>
+# include <link.h><br>
+#endif<br>
+<br>
+#define DW_LNS_copy 0x01<br>
+#define DW_LNS_advance_pc 0x02<br>
+#define DW_LNS_advance_line 0x03<br>
+#define DW_LNS_set_file 0x04<br>
+#define DW_LNS_set_column 0x05<br>
+#define DW_LNS_negate_stmt 0x06<br>
+#define DW_LNS_set_basic_block 0x07<br>
+#define DW_LNS_const_add_pc 0x08<br>
+#define DW_LNS_fixed_advance_pc 0x09<br>
+#define DW_LNS_set_prologue_end 0x0a /</em> DWARF3 <em>/<br>
+#define DW_LNS_set_epilogue_begin 0x0b /</em> DWARF3 <em>/<br>
+#define DW_LNS_set_isa 0x0c /</em> DWARF3 <em>/<br>
+<br>
+/</em> Line number extended opcode name. <em>/<br>
+#define DW_LNE_end_sequence 0x01<br>
+#define DW_LNE_set_address 0x02<br>
+#define DW_LNE_define_file 0x03<br>
+#define DW_LNE_set_discriminator 0x04 /</em> DWARF4 */<br>
+<br>
+# if SIZEOF_VOIDP == 8<br>
+# define ElfW(x) Elf64##<em>##x<br>
+# else<br>
+# define ElfW(x) Elf32##</em>##x<br>
+# endif<br>
+<br>
+typedef struct {</p>
<ul>
<li>const char *dirname;</li>
<li>const char *filename;</li>
<li>int line;</li>
<li>
<li>int fd;</li>
<li>void *mapped;</li>
<li>size_t mapped_size;</li>
<li>unsigned long base_addr;<br>
+} line_info_t;</li>
<li>
</ul>
<p>+/* Avoid consuming stack as this module may be used from signal handler */<br>
+static char binary_filename[PATH_MAX];<br>
+<br>
+static unsigned long<br>
+uleb128(char **p) {</p>
<ul>
<li>unsigned long r = 0;</li>
<li>int s = 0;</li>
<li>for (;;) {</li>
<li>unsigned char b = *(unsigned char *)(*p)++;</li>
<li>if (b < 0x80) {</li>
<li>
<pre><code> r += b << s;
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>}</li>
<li>r += (b & 0x7f) << s;</li>
<li>s += 7;</li>
<li>}</li>
<li>return r;<br>
+}</li>
<li>
</ul>
<p>+static long<br>
+sleb128(char **p) {</p>
<ul>
<li>long r = 0;</li>
<li>int s = 0;</li>
<li>for (;;) {</li>
<li>unsigned char b = *(unsigned char *)(*p)++;</li>
<li>if (b < 0x80) {</li>
<li>
<pre><code> if (b & 0x40) {
</code></pre>
</li>
<li>
<pre><code> r -= (0x80 - b) << s;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else {
</code></pre>
</li>
<li>
<pre><code> r += (b & 0x3f) << s;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>}</li>
<li>r += (b & 0x7f) << s;</li>
<li>s += 7;</li>
<li>}</li>
<li>return r;<br>
+}</li>
<li>
</ul>
<p>+static const char *<br>
+get_nth_dirname(int dir, char *p)<br>
+{</p>
<ul>
<li>if (!dir--) {</li>
<li>return "";</li>
<li>}</li>
<li>while (dir) {</li>
<li>while (*p) p++;</li>
<li>p++;</li>
<li>if (!*p) {</li>
<li>
<pre><code> fprintf(stderr, "Unexpected directory number %d in %s\n",
</code></pre>
</li>
<li>
<pre><code> dir, binary_filename);
</code></pre>
</li>
<li>
<pre><code> return "";
</code></pre>
</li>
<li>}</li>
<li>}</li>
<li>return p;<br>
+}</li>
<li>
</ul>
<p>+static void<br>
+fill_filename(int file, char *include_directories, char *filenames,</p>
<ul>
<li>
<pre><code> line_info_t *line)
</code></pre>
</li>
</ul>
<p>+{</p>
<ul>
<li>int i;</li>
<li>char *p = filenames;</li>
<li>char *filename;</li>
<li>unsigned long dir;</li>
<li>for (i = 1; i <= file; i++) {</li>
<li>filename = p;</li>
<li>if (!*p) {</li>
<li>
<pre><code> /* Need to output binary file name? */
</code></pre>
</li>
<li>
<pre><code> fprintf(stderr, "Unexpected file number %d in %s\n",
</code></pre>
</li>
<li>
<pre><code> file, binary_filename);
</code></pre>
</li>
<li>
<pre><code> return;
</code></pre>
</li>
<li>}</li>
<li>while (*p) p++;</li>
<li>p++;</li>
<li>dir = uleb128(&p);</li>
<li>/* last modified. */</li>
<li>uleb128(&p);</li>
<li>/* size of the file. */</li>
<li>uleb128(&p);</li>
<li>
<li>if (i == file) {</li>
<li>
<pre><code> line->filename = filename;
</code></pre>
</li>
<li>
<pre><code> line->dirname = get_nth_dirname(dir, include_directories);
</code></pre>
</li>
<li>}</li>
<li>}<br>
+}</li>
<li>
</ul>
<p>+static int<br>
+get_path_from_symbol(const char *symbol, const char **p, size_t *len)<br>
+{</p>
<ul>
<li>if (symbol[0] == '0') {</li>
<li>/* libexecinfo */</li>
<li>*p = strchr(symbol, '/');</li>
<li>if (*p == NULL) return 0;</li>
<li>*len = strlen(*p);</li>
<li>}</li>
<li>else {</li>
<li>/* glibc */</li>
<li>const char *q;</li>
<li>*p = symbol;</li>
<li>q = strchr(symbol, '(');</li>
<li>if (q == NULL) return 0;</li>
<li>*len = q - symbol;</li>
<li>}</li>
<li>return 1;<br>
+}</li>
<li>
</ul>
<p>+static void<br>
+fill_line(int num_traces, void **traces,</p>
<ul>
<li>unsigned long addr, int file, int line,</li>
<li>char *include_directories, char *filenames, line_info_t *lines)<br>
+{</li>
<li>int i;</li>
<li>for (i = 0; i < num_traces; i++) {</li>
<li>unsigned long a = (unsigned long)traces[i] - lines[i].base_addr;</li>
<li>/* We assume one line code doesn't result >100 bytes of native code.</li>
<li>
<pre><code> We may want more reliable way eventually... */
</code></pre>
</li>
<li>if (addr < a && a < addr + 100) {</li>
<li>
<pre><code> fill_filename(file, include_directories, filenames, &lines[i]);
</code></pre>
</li>
<li>
<pre><code> lines[i].line = line;
</code></pre>
</li>
<li>}</li>
<li>}<br>
+}</li>
<li>
</ul>
<p>+static void<br>
+parse_debug_line_cu(int num_traces, void **traces,</p>
<ul>
<li>
<pre><code> char **debug_line, line_info_t *lines)
</code></pre>
</li>
</ul>
<p>+{</p>
<ul>
<li>char *p, *cu_end, *cu_start, *include_directories, *filenames;</li>
<li>unsigned long unit_length;</li>
<li>int default_is_stmt, line_base;</li>
<li>unsigned int header_length, minimum_instruction_length, line_range,</li>
<li>
<pre><code> opcode_base;
</code></pre>
</li>
<li>unsigned char *standard_opcode_lengths;</li>
<li>
<li>/* The registers. */</li>
<li>unsigned long addr = 0;</li>
<li>unsigned int file = 1;</li>
<li>unsigned int line = 1;</li>
<li>unsigned int column = 0;</li>
<li>int is_stmt = default_is_stmt;</li>
<li>int basic_block = 0;</li>
<li>int end_sequence = 0;</li>
<li>int prologue_end = 0;</li>
<li>int epilogue_begin = 0;</li>
<li>unsigned int isa = 0;</li>
<li>
<li>p = *debug_line;</li>
<li>
<li>unit_length = *(unsigned int *)p;</li>
<li>p += sizeof(unsigned int);</li>
<li>if (unit_length == 0xffffffff) {</li>
<li>unit_length = *(unsigned long *)p;</li>
<li>p += sizeof(unsigned long);</li>
<li>}</li>
<li>
<li>cu_end = p + unit_length;</li>
<li>
<li>/*dwarf_version = *(unsigned short <em>)p;</em>/</li>
<li>p += 2;</li>
<li>
<li>header_length = *(unsigned int *)p;</li>
<li>p += sizeof(unsigned int);</li>
<li>
<li>cu_start = p + header_length;</li>
<li>
<li>minimum_instruction_length = *(unsigned char *)p;</li>
<li>p++;</li>
<li>
<li>default_is_stmt = *(unsigned char *)p;</li>
<li>p++;</li>
<li>
<li>line_base = *(char *)p;</li>
<li>p++;</li>
<li>
<li>line_range = *(unsigned char *)p;</li>
<li>p++;</li>
<li>
<li>opcode_base = *(unsigned char *)p;</li>
<li>p++;</li>
<li>
<li>standard_opcode_lengths = (unsigned char *)p - 1;</li>
<li>p += opcode_base - 1;</li>
<li>
<li>include_directories = p;</li>
<li>
<li>/* skip include directories */</li>
<li>while (*p) {</li>
<li>while (*p) p++;</li>
<li>p++;</li>
<li>}</li>
<li>p++;</li>
<li>
<li>filenames = p;</li>
<li>
<li>p = cu_start;</li>
<li>
</ul>
<p>+#define FILL_LINE() \</p>
<ul>
<li>do { \</li>
<li>fill_line(num_traces, traces, addr, file, line, \</li>
<li>
<pre><code> include_directories, filenames, lines); \
</code></pre>
</li>
<li>basic_block = prologue_end = epilogue_begin = 0; \</li>
<li>} while (0)</li>
<li>
<li>while (p < cu_end) {</li>
<li>unsigned long a;</li>
<li>unsigned char op = *p++;</li>
<li>switch (op) {</li>
<li>case DW_LNS_copy:</li>
<li>
<pre><code> FILL_LINE();
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>case DW_LNS_advance_pc:</li>
<li>
<pre><code> a = uleb128(&p);
</code></pre>
</li>
<li>
<pre><code> addr += a;
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>case DW_LNS_advance_line: {</li>
<li>
<pre><code> long a = sleb128(&p);
</code></pre>
</li>
<li>
<pre><code> line += a;
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>}</li>
<li>case DW_LNS_set_file:</li>
<li>
<pre><code> file = uleb128(&p);
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>case DW_LNS_set_column:</li>
<li>
<pre><code> column = uleb128(&p);
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>case DW_LNS_negate_stmt:</li>
<li>
<pre><code> is_stmt = !is_stmt;
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>case DW_LNS_set_basic_block:</li>
<li>
<pre><code> basic_block = 1;
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>case DW_LNS_const_add_pc:</li>
<li>
<pre><code> a = ((255 - opcode_base) / line_range) *
</code></pre>
</li>
<li>
<pre><code> minimum_instruction_length;
</code></pre>
</li>
<li>
<pre><code> addr += a;
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>case DW_LNS_fixed_advance_pc:</li>
<li>
<pre><code> a = *(unsigned char *)p++;
</code></pre>
</li>
<li>
<pre><code> addr += a;
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>case DW_LNS_set_prologue_end:</li>
<li>
<pre><code> prologue_end = 1;
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>case DW_LNS_set_epilogue_begin:</li>
<li>
<pre><code> epilogue_begin = 1;
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>case DW_LNS_set_isa:</li>
<li>
<pre><code> isa = uleb128(&p);
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>case 0:</li>
<li>
<pre><code> a = *(unsigned char *)p++;
</code></pre>
</li>
<li>
<pre><code> op = *p++;
</code></pre>
</li>
<li>
<pre><code> switch (op) {
</code></pre>
</li>
<li>
<pre><code> case DW_LNE_end_sequence:
</code></pre>
</li>
<li>
<pre><code> end_sequence = 1;
</code></pre>
</li>
<li>
<pre><code> FILL_LINE();
</code></pre>
</li>
<li>
<pre><code> addr = 0;
</code></pre>
</li>
<li>
<pre><code> file = 1;
</code></pre>
</li>
<li>
<pre><code> line = 1;
</code></pre>
</li>
<li>
<pre><code> column = 0;
</code></pre>
</li>
<li>
<pre><code> is_stmt = default_is_stmt;
</code></pre>
</li>
<li>
<pre><code> end_sequence = 0;
</code></pre>
</li>
<li>
<pre><code> isa = 0;
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>
<pre><code> case DW_LNE_set_address:
</code></pre>
</li>
<li>
<pre><code> addr = *(unsigned long *)p;
</code></pre>
</li>
<li>
<pre><code> p += sizeof(unsigned long);
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>
<pre><code> case DW_LNE_define_file:
</code></pre>
</li>
<li>
<pre><code> fprintf(stderr, "Unsupported operation in %s\n",
</code></pre>
</li>
<li>
<pre><code> binary_filename);
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>
<pre><code> default:
</code></pre>
</li>
<li>
<pre><code> fprintf(stderr, "Unknown extended opcode: %d in %s\n",
</code></pre>
</li>
<li>
<pre><code> op, binary_filename);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>default: {</li>
<li>
<pre><code> unsigned int addr_incr;
</code></pre>
</li>
<li>
<pre><code> int line_incr;
</code></pre>
</li>
<li>
<pre><code> a = op - opcode_base;
</code></pre>
</li>
<li>
<pre><code> addr_incr = (a / line_range) * minimum_instruction_length;
</code></pre>
</li>
<li>
<pre><code> line_incr = line_base + (a % line_range);
</code></pre>
</li>
<li>
<pre><code> addr += addr_incr;
</code></pre>
</li>
<li>
<pre><code> line += line_incr;
</code></pre>
</li>
<li>
<pre><code> FILL_LINE();
</code></pre>
</li>
<li>}</li>
<li>}</li>
<li>}</li>
<li>*debug_line = p;<br>
+}</li>
<li>
</ul>
<p>+static void<br>
+parse_debug_line(int num_traces, void **traces,</p>
<ul>
<li>
<pre><code> char *debug_line, unsigned long size, line_info_t *lines)
</code></pre>
</li>
</ul>
<p>+{</p>
<ul>
<li>char *debug_line_end = debug_line + size;</li>
<li>while (debug_line < debug_line_end) {</li>
<li>parse_debug_line_cu(num_traces, traces, &debug_line, lines);</li>
<li>}</li>
<li>if (debug_line != debug_line_end) {</li>
<li>fprintf(stderr, "Unexpected size of .debug_line in %s\n",</li>
<li>
<pre><code> binary_filename);
</code></pre>
</li>
<li>}<br>
+}</li>
<li>
</ul>
<p>+/* read file and fill lines */<br>
+static void<br>
+fill_lines(int num_traces, void **traces, char **syms,</p>
<ul>
<li>
<pre><code>char *file, line_info_t *lines)
</code></pre>
</li>
</ul>
<p>+{</p>
<ul>
<li>int i;</li>
<li>char *shstr;</li>
<li>char *section_name;</li>
<li>ElfW(Ehdr) *ehdr;</li>
<li>ElfW(Shdr) *shdr, *shstr_shdr, *debug_line_shdr = NULL;</li>
<li>
<li>for (i = 0; i < num_traces; i++) {</li>
<li>const char *path;</li>
<li>size_t len;</li>
<li>if (get_path_from_symbol(syms[i], &path, &len) &&</li>
<li>
<pre><code> !strncmp(path, binary_filename, len)) {
</code></pre>
</li>
<li>
<pre><code> lines[i].line = -1;
</code></pre>
</li>
<li>}</li>
<li>}</li>
<li>
<li>ehdr = (ElfW(Ehdr) *)file;</li>
<li>shdr = (ElfW(Shdr) *)(file + ehdr->e_shoff);</li>
<li>
<li>shstr_shdr = shdr + ehdr->e_shstrndx;</li>
<li>shstr = file + shstr_shdr->sh_offset;</li>
<li>
<li>for (i = 0; i < ehdr->e_shnum; i++) {</li>
<li>section_name = shstr + shdr[i].sh_name;</li>
<li>if (!strcmp(section_name, ".debug_line")) {</li>
<li>
<pre><code> debug_line_shdr = shdr + i;
</code></pre>
</li>
<li>
<pre><code> break;
</code></pre>
</li>
<li>}</li>
<li>}</li>
<li>
<li>if (!debug_line_shdr) {</li>
<li>/* this file doesn't have .debug_line section */</li>
<li>return;</li>
<li>}</li>
<li>
<li>parse_debug_line(num_traces, traces,</li>
<li>
<pre><code> file + debug_line_shdr->sh_offset,
</code></pre>
</li>
<li>
<pre><code> debug_line_shdr->sh_size,
</code></pre>
</li>
<li>
<pre><code> lines);
</code></pre>
</li>
</ul>
<p>+}<br>
+<br>
+#ifdef HAVE_DL_ITERATE_PHDR<br>
+<br>
+typedef struct {</p>
<ul>
<li>int num_traces;</li>
<li>char **syms;</li>
<li>line_info_t *lines;<br>
+} fill_base_addr_state_t;</li>
<li>
</ul>
<p>+static int<br>
+fill_base_addr(struct dl_phdr_info *info, size_t size, void *data)<br>
+{</p>
<ul>
<li>int i;</li>
<li>fill_base_addr_state_t *st = (fill_base_addr_state_t *)data;</li>
<li>for (i = 0; i < st->num_traces; i++) {</li>
<li>const char *path;</li>
<li>size_t len;</li>
<li>size_t name_len = strlen(info->dlpi_name);</li>
<li>
<li>if (get_path_from_symbol(st->syms[i], &path, &len) &&</li>
<li>
<pre><code> (len == name_len || (len > name_len && path[len-name_len-1] == '/')) &&
</code></pre>
</li>
<li>
<pre><code> !strncmp(path+len-name_len, info->dlpi_name, name_len)) {
</code></pre>
</li>
<li>
<pre><code> st->lines[i].base_addr = info->dlpi_addr;
</code></pre>
</li>
<li>}</li>
<li>}</li>
<li>return 0;<br>
+}</li>
<li>
</ul>
<p>+#endif /* HAVE_DL_ITERATE_PHDR */<br>
+<br>
+void<br>
+rb_dump_backtrace_with_lines(int num_traces, void **trace, char **syms)<br>
+{</p>
<ul>
<li>int i;</li>
<li>int fd;</li>
<li>/* async-signal unsafe */</li>
<li>line_info_t *lines = (line_info_t *)calloc(num_traces,</li>
<li>
<pre><code> sizeof(line_info_t));
</code></pre>
</li>
<li>off_t filesize;</li>
<li>char *file;</li>
<li>
<li>/* Note that line info of shared objects might not be shown</li>
<li>
<pre><code> if we don't have dl_iterate_phdr */
</code></pre>
</li>
</ul>
<p>+#ifdef HAVE_DL_ITERATE_PHDR</p>
<ul>
<li>fill_base_addr_state_t fill_base_addr_state;</li>
<li>
<li>fill_base_addr_state.num_traces = num_traces;</li>
<li>fill_base_addr_state.syms = syms;</li>
<li>fill_base_addr_state.lines = lines;</li>
<li>/* maybe async-signal unsafe */</li>
<li>dl_iterate_phdr(fill_base_addr, &fill_base_addr_state);<br>
+#endif /* HAVE_DL_ITERATE_PHDR */</li>
<li>
<li>for (i = 0; i < num_traces; i++) {</li>
<li>const char *path;</li>
<li>size_t len;</li>
<li>if (lines[i].line) {</li>
<li>
<pre><code> continue;
</code></pre>
</li>
<li>}</li>
<li>
<li>if (!get_path_from_symbol(syms[i], &path, &len)) {</li>
<li>
<pre><code> continue;
</code></pre>
</li>
<li>}</li>
<li>
<li>strncpy(binary_filename, path, len);</li>
<li>binary_filename[len] = '\0';</li>
<li>
<li>fd = open(binary_filename, O_RDONLY);</li>
<li>filesize = lseek(fd, 0, SEEK_END);</li>
<li>lseek(fd, 0, SEEK_SET);</li>
<li>/* async-signal unsafe */</li>
<li>file = (char *)mmap(NULL, filesize, PROT_READ, MAP_SHARED, fd, 0);</li>
<li>
<li>lines[i].fd = fd;</li>
<li>lines[i].mapped = file;</li>
<li>lines[i].mapped_size = filesize;</li>
<li>
<li>fill_lines(num_traces, trace, syms, file, lines);</li>
<li>}</li>
<li>
<li>/* fprintf may not be async-signal safe */</li>
<li>for (i = 0; i < num_traces; i++) {</li>
<li>line_info_t *line = &lines[i];</li>
<li>
<li>if (line->line > 0) {</li>
<li>
<pre><code> fprintf(stderr, "%s ", syms[i]);
</code></pre>
</li>
<li>
<pre><code> if (line->filename) {
</code></pre>
</li>
<li>
<pre><code> if (line->dirname && line->dirname[0]) {
</code></pre>
</li>
<li>
<pre><code> fprintf(stderr, "%s/", line->dirname);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> fprintf(stderr, "%s", line->filename);
</code></pre>
</li>
<li>
<pre><code> } else {
</code></pre>
</li>
<li>
<pre><code> fprintf(stderr, "???");
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> fprintf(stderr, ":%d\n", line->line);
</code></pre>
</li>
<li>} else {</li>
<li>
<pre><code> fprintf(stderr, "%s\n", syms[i]);
</code></pre>
</li>
<li>}</li>
<li>}</li>
<li>
<li>for (i = 0; i < num_traces; i++) {</li>
<li>line_info_t *line = &lines[i];</li>
<li>if (line->fd) {</li>
<li>
<pre><code> munmap(line->mapped, line->mapped_size);
</code></pre>
</li>
<li>
<pre><code> close(line->fd);
</code></pre>
</li>
<li>}</li>
<li>}</li>
<li>free(lines);<br>
+}</li>
<li>
</ul>
<p>+#endif /* defined(<strong>ELF</strong>) <em>/<br>
diff --git a/addr2line.h b/addr2line.h<br>
new file mode 100644<br>
index 0000000..cbb18e5<br>
--- /dev/null<br>
+++ b/addr2line.h<br>
@@ -0,0 +1,21 @@<br>
+/</em>*********************************************************************<br>
+</p>
<ul>
<li>addr2line.h -</li>
<li>
<li>$Author$</li>
<li>
<li>Copyright (C) 2010 Shinichiro Hamaji</li>
<li>
</ul>
<p>+**********************************************************************/<br>
+<br>
+#ifndef RUBY_ADDR2LINE_H<br>
+#define RUBY_ADDR2LINE_H<br>
+<br>
+#ifdef <strong>ELF</strong><br>
+<br>
+void<br>
+rb_dump_backtrace_with_lines(int num_traces, void **traces, char *<em>syms);<br>
+<br>
+#endif /</em> <strong>ELF</strong> <em>/<br>
+<br>
+#endif /</em> RUBY_ADDR2LINE_H */<br>
diff --git a/common.mk b/common.mk<br>
index c45c3e1..cf66e42 100644<br>
--- a/common.mk<br>
+++ b/common.mk<br>
@@ -91,6 +91,7 @@ COMMONOBJS = array.$(OBJEXT) <br>
vm_dump.$(OBJEXT) <br>
thread.$(OBJEXT) <br>
cont.$(OBJEXT) \</p>
<ul>
<li>
<pre><code> addr2line.$(OBJEXT) \
$(BUILTIN_ENCOBJS) \
$(BUILTIN_TRANSOBJS) \
$(MISSING)
</code></pre>
</li>
</ul>
<p>diff --git a/configure.in b/configure.in<br>
index 36a58d4..ef2ee2d 100644<br>
--- a/configure.in<br>
+++ b/configure.in<br>
@@ -1300,7 +1300,7 @@ AC_CHECK_FUNCS(fmod killpg wait4 waitpid fork spawnv syscall chroot getcwd eacce<br>
setsid telldir seekdir fchmod cosh sinh tanh log2 round<br>
setuid setgid daemon select_large_fdset setenv unsetenv<br>
mktime timegm gmtime_r clock_gettime gettimeofday\</p>
<ul>
<li>
<pre><code> pread sendfile shutdown sigaltstack)
</code></pre>
</li>
</ul>
<ul>
<li>
<pre><code> pread sendfile shutdown sigaltstack dl_iterate_phdr)
</code></pre>
</li>
</ul>
<p>AC_CACHE_CHECK(for unsetenv returns a value, rb_cv_unsetenv_return_value,<br>
[AC_TRY_COMPILE([<br>
diff --git a/vm_dump.c b/vm_dump.c<br>
index 2975001..b22c041 100644<br>
--- a/vm_dump.c<br>
+++ b/vm_dump.c<br>
@@ -10,6 +10,7 @@</p>
<p>#include "ruby/ruby.h"<br>
+#include "addr2line.h"<br>
#include "vm_core.h"</p>
<p>#define MAX_POSBUF 128<br>
@@ -785,9 +786,13 @@ rb_vm_bugreport(void)<br>
int i;</p>
<pre><code>if (syms) {
</code></pre>
<p>+#ifdef <strong>ELF</strong></p>
<ul>
<li>
<pre><code> rb_dump_backtrace_with_lines(n, trace, syms);
</code></pre>
</li>
</ul>
<p>+#else<br>
for (i=0; i<n; i++) {<br>
fprintf(stderr, "%s\n", syms[i]);<br>
}<br>
+#endif<br>
free(syms);<br>
}<br>
#elif defined(_WIN32)<br>
=end</p>
Ruby master - Feature #3961 (Rejected): printfと精度指定と負の値と
https://redmine.ruby-lang.org/issues/3961
2010-10-18T18:01:01Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
printf が、フォーマットに精度指定しつつ負の値を与えた時の挙動が、<br>
Perl と必然性無く異なっていて不便です。</p>
<p>% ./ruby -e'printf("%#.8x", -1)'<br>
0x..ffffff</p>
<p>そもそもこの挙動は C 言語由来で、C の場合例えば以下の通りになります。</p>
<p>% cat test.c<br>
#include <stdio.h><br>
int main(void)<br>
{<br>
printf("%#x\n", -1);<br>
return 0;<br>
}<br>
% cc t.c && ./a.out<br>
0xffffffff</p>
<p>また、おそらく直接参考にしたであろう Perl では以下の通りです、<br>
% perl -e'printf("%#x",-1)'<br>
0xffffffffffffffff<br>
% perl -e'printf("%#.30x",-1)'<br>
0x00000000000000ffffffffffffffff<br>
つまり、Perl (の 64bit int 版) では、64bitで補数を取っています。</p>
<p>さて、Ruby の場合多倍長整数が組み込みなため、補数を取ると無限に続いてしまうから、<br>
% ./ruby -e'printf("%#x",-1)'<br>
0x..f<br>
と .. で略すのは理にかなっていると思います。</p>
<p>しかし、現在の Ruby は精度を指定した際にも..がついてしまうので、<br>
Perl のような動きを実現させる事ができません。<br>
% ./ruby -e'printf("%#.8x",-1)'<br>
0x..ffffff<br>
で、この .. っていらないと思うんです。</p>
<p>例えば浮動小数点数の場合には以下のように .. とか付けずにぶった切る訳で、<br>
無限大方向と無限小方向という違いはあれど、<br>
「Perlとかと同じ挙動にできない」というデメリットの方が大きいのではないでしょうか。<br>
% ./ruby -e'printf("%#.8f",10.0/3)'<br>
3.33333333</p>
<p>わたしの場合、CRuby 側の inspect が printf("%x", negative_value) などとしている部分の動作を<br>
RubySpec で Ruby で書く時に頭を抱えてしまいました。</p>
<p>というわけでパッチは例えば以下の通りです。</p>
<p>diff --git a/sprintf.c b/sprintf.c<br>
index 21509ea..0e97955 100644<br>
--- a/sprintf.c<br>
+++ b/sprintf.c<br>
@@ -844,7 +844,7 @@ rb_str_format(int argc, const VALUE *argv, VALUE fmt)<br>
}<br>
else {<br>
s = nbuf;</p>
<ul>
<li>
<pre><code> if (v < 0) {
</code></pre>
</li>
</ul>
<ul>
<li>
<pre><code> if (v < 0 && !(flags & FPREC0)) {
dots = 1;
}
snprintf(fbuf, sizeof(fbuf), "%%l%c", *p == 'X' ? 'x' : *p);
</code></pre>
</li>
</ul>
<p>@@ -892,7 +892,8 @@ rb_str_format(int argc, const VALUE *argv, VALUE fmt)<br>
tmp1 = tmp = rb_big2str0(val, base, RBIGNUM_SIGN(val));<br>
s = RSTRING_PTR(tmp);<br>
if (*s == '-') {</p>
<ul>
<li>
<pre><code> dots = 1;
</code></pre>
</li>
</ul>
<ul>
<li>
<pre><code> if (!(flags & FPREC0))
</code></pre>
</li>
<li>
<pre><code> dots = 1;
if (base == 10) {
rb_warning("negative number for %%u specifier");
}
</code></pre>
</li>
</ul>
<p>@@ -925,14 +926,11 @@ rb_str_format(int argc, const VALUE <em>argv, VALUE fmt)<br>
}<br>
}<br>
if (prefix && !prefix[1]) { /</em> octal */</p>
<ul>
<li>
<pre><code> if (dots) {
</code></pre>
</li>
<li>
<pre><code> prefix = 0;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> else if (len == 1 && *s == '0') {
</code></pre>
</li>
</ul>
<ul>
<li>
<pre><code> if (len == 1 && *s == '0') {
len = 0;
if (flags & FPREC) prec--;
}
</code></pre>
</li>
</ul>
<ul>
<li>
<pre><code> else if ((flags & FPREC) && (prec > len)) {
</code></pre>
</li>
</ul>
<ul>
<li>
<pre><code> else if ((flags & FPREC) && (prec > len) && v >= 0) {
prefix = 0;
}
}
</code></pre>
</li>
</ul>
<p>diff --git a/test/ruby/test_sprintf.rb b/test/ruby/test_sprintf.rb<br>
index 96a1b62..24b6a34 100644<br>
--- a/test/ruby/test_sprintf.rb<br>
+++ b/test/ruby/test_sprintf.rb<br>
@@ -24,12 +24,12 @@ class TestSprintf < Test::Unit::TestCase<br>
assert_equal("0000", sprintf("%.4b", 0))<br>
assert_equal("0001", sprintf("%.4b", 1))<br>
assert_equal("0010", sprintf("%.4b", 2))</p>
<ul>
<li>assert_equal("..11", sprintf("%.4b", -1))</li>
</ul>
<ul>
<li>
<p>assert_equal("1111", sprintf("%.4b", -1))</p>
<p>assert_equal(" 0000", sprintf("%6.4b", 0))<br>
assert_equal(" 0001", sprintf("%6.4b", 1))<br>
assert_equal(" 0010", sprintf("%6.4b", 2))</p>
</li>
</ul>
<ul>
<li>assert_equal(" ..11", sprintf("%6.4b", -1))</li>
</ul>
<ul>
<li>
<p>assert_equal(" 1111", sprintf("%6.4b", -1))</p>
<p>assert_equal(" 0", sprintf("%#4b", 0))<br>
assert_equal(" 0b1", sprintf("%#4b", 1))<br>
@@ -44,12 +44,12 @@ class TestSprintf < Test::Unit::TestCase<br>
assert_equal("0000", sprintf("%#.4b", 0))<br>
assert_equal("0b0001", sprintf("%#.4b", 1))<br>
assert_equal("0b0010", sprintf("%#.4b", 2))</p>
</li>
</ul>
<ul>
<li>assert_equal("0b..11", sprintf("%#.4b", -1))</li>
</ul>
<ul>
<li>
<p>assert_equal("0b1111", sprintf("%#.4b", -1))</p>
<p>assert_equal(" 0000", sprintf("%#6.4b", 0))<br>
assert_equal("0b0001", sprintf("%#6.4b", 1))<br>
assert_equal("0b0010", sprintf("%#6.4b", 2))</p>
</li>
</ul>
<ul>
<li>assert_equal("0b..11", sprintf("%#6.4b", -1))</li>
</ul>
<ul>
<li>
<p>assert_equal("0b1111", sprintf("%#6.4b", -1))</p>
<p>assert_equal("+0", sprintf("%+b", 0))<br>
assert_equal("+1", sprintf("%+b", 1))<br>
@@ -288,6 +288,8 @@ class TestSprintf < Test::Unit::TestCase<br>
b1 = (/../ =~ s1) != nil<br>
b2 = (/../ =~ s2) != nil<br>
assert(b1 == b2, "<a href="https://blade.ruby-lang.org/ruby-dev/33224">[ruby-dev:33224]</a>")</p>
</li>
<li>
<li>
<p>assert_equal("ffffffff", sprintf("%.8x", -1))<br>
end</p>
</li>
</ul>
<pre><code>def test_named
</code></pre>
<p>diff --git a/test/ruby/test_sprintf_comb.rb b/test/ruby/test_sprintf_comb.rb<br>
index 261732b..3105127 100644<br>
--- a/test/ruby/test_sprintf_comb.rb<br>
+++ b/test/ruby/test_sprintf_comb.rb<br>
@@ -190,7 +190,7 @@ class TestSprintfComb < Test::Unit::TestCase<br>
if digits.last != radix-1<br>
digits << (radix-1)<br>
end</p>
<ul>
<li>
<pre><code> sign = '..'
</code></pre>
</li>
</ul>
<ul>
<li>
<pre><code> sign = '..' unless precision
else
sign = '-'
end
</code></pre>
</li>
</ul>
<p>@@ -222,8 +222,8 @@ class TestSprintfComb < Test::Unit::TestCase<br>
end<br>
end<br>
if type == 'o' && hs</p>
<ul>
<li>
<pre><code> if digits.empty? || digits.last != d
</code></pre>
</li>
<li>
<pre><code> digits << d
</code></pre>
</li>
</ul>
<ul>
<li>
<pre><code> if digits.empty? || digits.last != 0
</code></pre>
</li>
<li>
<pre><code> prefix = '0'
end
</code></pre>
end<br>
=end</li>
</ul>
Ruby master - Feature #3947 (Closed): Array#packのにエンディアン指定修飾子</>を追加
https://redmine.ruby-lang.org/issues/3947
2010-10-14T15:46:42Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
Ruby の伏魔殿 Array#pack ですが、機種依存な部分をいじる際にはしばしば強力かつ唯一の手段になる事があります。<br>
具体的には RubySpec 書く時とか。</p>
<p>さて、pack のようにバイト列を扱う場合、しばしば問題になるのがエンディアンです。<br>
ここで、Ruby の Array#pack は 16bit/32bit 固定の整数に対してはエンディアン固定の<br>
n/N/v/V 指定子を用意していますが、short/int/long/long long のように、<br>
マシン依存の型をエンディアン固定で pack する手段を提供していません。<br>
というわけで、これが欲しいです。</p>
<p>ここで問題になるのがどのように指定するかなのですが、pack のネタ元である Perl さんでは既に </> 修飾子を<br>
この目的のために提供しています。<br>
<a href="http://perldoc.perl.org/functions/pack.html" class="external">http://perldoc.perl.org/functions/pack.html</a><br>
というわけで、これに追従するのがよろしいと思います。</p>
<pre><code> > sSiIlLqQ Force big-endian byte-order on the type.
jJfFdDpP (The "big end" touches the construct.)
< sSiIlLqQ Force little-endian byte-order on the type.
jJfFdDpP (The "little end" touches the construct.)
</code></pre>
<p>なお、「</> とかきもい」という意見もあるかとは思いますが、pack は機種依存とかに触るきもいメソッドなので、<br>
そこを気にするよりは長い物に巻かれておいた方が無難ではないかなと思うため、</> 修飾子がベストかと思います。<br>
=end</p>
Ruby master - Feature #3946 (Closed): Array#packのqQ指定子に機種依存サイズフラグ!を追加
https://redmine.ruby-lang.org/issues/3946
2010-10-14T15:36:56Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
Ruby の伏魔殿 Array#pack ですが、機種依存な部分をいじる際にはしばしば強力かつ唯一の手段になる事があります。<br>
具体的には RubySpec 書く時とか。</p>
<p>さて、q/Q は 64bit signed/unsigned int なのですが、機種依存の long long (厳密には LONG_LONG) のサイズが必要な場合、<br>
現状 Ruby レベルから取る手段が一切ありません。<br>
すでに s/S/i/I/l/L (16bit/32bit/32bit) は、! をつけることでその機種の short/int/long のサイズになるため、<br>
これを q/Q に拡張すると唯一の手段が提供される事になります。</p>
<p>というわけで、Array#packのqQ指定子に機種依存サイズフラグ!を追加したいです。<br>
=end</p>
Backport186 - Backport #3403 (Closed): A bug related to ruby's regular expression!!!!!!
https://redmine.ruby-lang.org/issues/3403
2010-06-07T16:01:37Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
Backport r28192</p>
<p>I found a bug related to ruby's regular expression!<br>
puts(/wo{0,3}?/.match("woo"))<br>
This line of code should show a "w" on the screen, but my ruby shows "wo" on then screen.<br>
According to the principles of regular expression, this is a non-greedy match, so it should match 0 "o", but it matches at least 1 "o". I think this is a bug!<br>
I also tried this in the javascript language, it works well!("w" is shown)</p>
<p>My ruby version is v1.8.7-p249.<br>
=end</p>
Backport187 - Backport #3402 (Closed): A bug related to ruby's regular expression!!!!!!
https://redmine.ruby-lang.org/issues/3402
2010-06-07T16:00:55Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
Backport r28192</p>
<p>I found a bug related to ruby's regular expression!<br>
puts(/wo{0,3}?/.match("woo"))<br>
This line of code should show a "w" on the screen, but my ruby shows "wo" on then screen.<br>
According to the principles of regular expression, this is a non-greedy match, so it should match 0 "o", but it matches at least 1 "o". I think this is a bug!<br>
I also tried this in the javascript language, it works well!("w" is shown)</p>
<p>My ruby version is v1.8.7-p249.<br>
=end</p>
Ruby master - Feature #3036 (Closed): String#encode(to, from, opt) の opt[:replace] に Hash
https://redmine.ruby-lang.org/issues/3036
2010-03-29T16:11:29Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
<a href="https://blade.ruby-lang.org/ruby-dev/40540">[ruby-dev:40540]</a> の String#encode(to, from, opt) の opt[:replace] に Hash の話を、<br>
別にチケットとして起こしておきます</p>
<p>String#encode(to, from, opt) の opt[:replace] に Hash を与えられるようにして、<br>
そこに、変換元 encoding の文字 => 変換先の文字、という未定義文字の fallback 変換表を与えられるようにする、<br>
というものを今考えています。<br>
これだと例えば、<br>
fallbacks = {<br>
?¥uE6AD => "[ふくろ]",<br>
?¥u{1F4BA} => "[いす]"<br>
}<br>
"¥u{3042 E6AD 1F4BA}".encode("UTF8-KDDI", replace: fallbacks) #=> "あ[ふくろ][いす]"<br>
=end</p>
Ruby master - Feature #2969 (Closed): String#to_f が -h.hhh±pd を解釈できるように
https://redmine.ruby-lang.org/issues/2969
2010-03-16T04:01:03Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
C99 の printf には a という指定子があります。</p>
<pre><code> aA The argument is printed in style ‘[-h.hhh±pd]’ where there is
one digit before the hexadecimal point and the number after
is equal to the precision specification for the argument;
when the precision is missing, enough digits are produced to
convey the argument's exact double-precision floating-point
representation. The values ∞ and NaN are printed as ‘inf’
and ‘nan’, respectively.
</code></pre>
<p>これを使うと、以下のような形で整形されます。<br>
-0.0 #=> "-0x0p+0"<br>
729.0/10 #=> "0x1.239999999999ap+6"<br>
Math.log(3) #=> "0x1.193ea7aad030ap+0"<br>
Math.exp(100) #=> "0x1.3494a9b171bf5p+144"</p>
<p>この形式の利点は、複雑な浮動小数点数を比較的少ない文字数で正確に表せることと、<br>
仮数部が16進であるため丸めが起きていることを説明する際に便利な点が挙げられます。</p>
<p>で、この形式を使っているのですが、RubyのString#to_f で解釈してくれず悲しくなるので、<br>
解釈できるようにしませんか。<br>
パッチは以下の通りです。</p>
<p>diff --git a/util.c b/util.c<br>
index 5ebc5f3..e361d51 100644<br>
--- a/util.c<br>
+++ b/util.c<br>
@@ -2106,6 +2106,44 @@ ruby_strtod(const char *s00, char **se)<br>
}<br>
break2:<br>
if (*s == '0') {</p>
<ul>
<li>
<pre><code> if (s[1] == 'x') {
</code></pre>
</li>
<li>
<pre><code> static const char hexdigit[] = "0123456789abcdef0123456789ABCDEF";
</code></pre>
</li>
<li>
<pre><code> s0 = ++s;
</code></pre>
</li>
<li>
<pre><code> adj = 0;
</code></pre>
</li>
<li>
<li>
<pre><code> while (*++s && (s1 = strchr(hexdigit, *s))) {
</code></pre>
</li>
<li>
<pre><code> adj *= 16;
</code></pre>
</li>
<li>
<pre><code> adj += (s1 - hexdigit) & 15;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<li>
<pre><code> if (*s == '.') {
</code></pre>
</li>
<li>
<pre><code> aadj = 1.;
</code></pre>
</li>
<li>
<pre><code> while (*++s && (s1 = strchr(hexdigit, *s))) {
</code></pre>
</li>
<li>
<pre><code> aadj /= 16;
</code></pre>
</li>
<li>
<pre><code> adj += aadj * ((s1 - hexdigit) & 15);
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<li>
<pre><code> if (*s != 'p') {
</code></pre>
</li>
<li>
<pre><code> s = s0;
</code></pre>
</li>
<li>
<pre><code> goto ret;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<li>
<pre><code> dsign = 0x2C - *++s; /* +: 2B, -: 2D */
</code></pre>
</li>
<li>
<pre><code> if (abs(dsign) != 1) {
</code></pre>
</li>
<li>
<pre><code> s = s0;
</code></pre>
</li>
<li>
<pre><code> goto ret;
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<li>
<pre><code> for (nd = 0, s++; (c = *s) >= '0' && c <= '9'; s++) {
</code></pre>
</li>
<li>
<pre><code> nd *= 10;
</code></pre>
</li>
<li>
<pre><code> nd += c;
</code></pre>
</li>
<li>
<pre><code> nd -= '0';
</code></pre>
</li>
<li>
<pre><code> }
</code></pre>
</li>
<li>
<li>
<pre><code> dval(rv) = ldexp(adj, nd * dsign);
</code></pre>
</li>
<li>
<pre><code> goto ret;
</code></pre>
</li>
<li>
<pre><code> }
nz0 = 1;
while (*++s == '0') ;
if (!*s)
</code></pre>
</li>
</ul>
<p>diff --git a/test/ruby/test_string.rb b/test/ruby/test_string.rb<br>
index 64205f6..72d3242 100644<br>
--- a/test/ruby/test_string.rb<br>
+++ b/test/ruby/test_string.rb<br>
@@ -1381,10 +1381,24 @@ class TestString < Test::Unit::TestCase<br>
end</p>
<pre><code>def test_to_f
</code></pre>
<ul>
<li>assert_equal(0.0, S("0.0").to_f)</li>
<li>assert_equal(?0, S("0.0").to_f.to_s[0])</li>
<li>assert_equal(-0.0, S("-0.0").to_f)</li>
<li>assert_equal(?-, S("-0.0").to_f.to_s[0])<br>
assert_equal(344.3, S("344.3").to_f)<br>
assert_equal(5.9742e24, S("5.9742e24").to_f)<br>
assert_equal(98.6, S("98.6 degrees").to_f)<br>
assert_equal(0.0, S("degrees 100.0").to_f)</li>
<li>assert_equal(0.0, S("0x0p+0").to_f)</li>
<li>assert_equal(?0, S("0x0p+0").to_f.to_s[0])</li>
<li>assert_equal(-0.0, S("-0x0p+0").to_f)</li>
<li>assert_equal(?-, S("-0x0p+0").to_f.to_s[0])</li>
<li>assert_equal(1.0, S("0x1p+0").to_f)</li>
<li>assert_equal(?1, S("0x1p+0").to_f.to_s[0])</li>
<li>assert_equal(1024.0, S("0x1p+10").to_f)</li>
<li>assert_equal(0.0009765625, S("0x1p-10").to_f)</li>
<li>assert_equal(2.6881171418161356e+43, S("0x1.3494a9b171bf5p+144").to_f)</li>
<li>assert_equal(-3.720075976020836e-44, S("-0x1.a8c1f14e2af5dp-145").to_f)<br>
end</li>
</ul>
<pre><code>def test_to_i
</code></pre>
<p>=end</p>
Ruby master - Feature #2968 (Rejected): 数値の正負を返すメソッド
https://redmine.ruby-lang.org/issues/2968
2010-03-16T03:38:54Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
数値の正負を返すメソッドが欲しいです。<br>
主たる想定用途は 0.0 と -0.0 を区別したいときです。<br>
これは、0.0 > -0.0 や 0.0 == -0.0 では知ることができません。<br>
とりあえず flo.to_s[0] == ?- で知ることができますが、これではあんまりです。</p>
<p>悩みどころはいつもの通りメソッド名ですが、</p>
<ul>
<li>Numeric#positive? と Numeric#negative?</li>
<li>Numeric#sign -> 負で -1、正で 1</li>
<li>Numeric#sign? -> 負で true, 正で false (signbit(3) に習う)<br>
あたりでしょうか。</li>
</ul>
<p>いかがでしょう。<br>
=end</p>
Ruby master - Feature #2835 (Closed): String#encodeに置換文字列のみを与えたら未定義向けとみなす
https://redmine.ruby-lang.org/issues/2835
2010-03-03T21:53:11Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
Ruby 1.9 もおおむね完成してからそろそろ 3 年が経つのですが、<br>
気付いた事として、String#encode(to, from, opt) の opt に :replace を指定した時は、<br>
ほとんどの場合 :undef => :replace も指定するという事があります。</p>
<p>よくよく考えれば、不正なバイト列を続行したいという事は少ないし、推奨されず、<br>
ゆえに置換文字列を指定する場合はたいてい未定義文字を置き換えたい場合な訳です。</p>
<p>以下のパッチは、:invalid => :replace がしているされている場合以外に、<br>
(つまり、不正なバイト列向けに置換文字列が指定されている可能性がある場合以外)<br>
置換文字列を指定している場合、:undef => :replace とみなします。</p>
<p>想定ケースでは :undef => :replace と書かなくてすむので楽な上に、<br>
楽な方法に誘導する事で、よく考えずに {:invalid => :replace, :undef => :replace}<br>
と書いてしまう人を減らせるので一石二鳥だと思うのですが、いかがでしょう。</p>
<p>diff --git a/transcode.c b/transcode.c<br>
index d511547..9d6a886 100644<br>
--- a/transcode.c<br>
+++ b/transcode.c<br>
@@ -2394,6 +2394,11 @@ econv_opts(VALUE opt)<br>
rb_raise(rb_eArgError, "unknown value for undefined character option");<br>
}</p>
<ul>
<li>v = rb_hash_aref(opt, sym_replace);</li>
<li>if (!NIL_P(v) && !(ecflags & ECONV_INVALID_REPLACE)) {</li>
<li>
<pre><code> ecflags |= ECONV_UNDEF_REPLACE;
</code></pre>
</li>
<li>}</li>
<li>v = rb_hash_aref(opt, sym_xml);<br>
if (!NIL_P(v)) {<br>
if (v==sym_text) {<br>
=end</li>
</ul>
Ruby master - Feature #2635 (Rejected): Unbundle rdoc
https://redmine.ruby-lang.org/issues/2635
2010-01-23T23:20:56Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
The rdoc in trunk is outdated and not maintained,<br>
and latest rdoc is in gen.</p>
<p>I think Ruby 1.9 shouldn't bundle such old rdoc.<br>
People who needs rdoc should install from gem.<br>
=end</p>
Ruby master - Feature #2579 (Closed): Net::HTTP.start("www.ruby-lang.org", use_ssl: true) で SSL 利...
https://redmine.ruby-lang.org/issues/2579
2010-01-09T20:34:21Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
従来、Net::HTTP で https アクセスをするためには</p>
<p>http = Net::HTTP.new("www.ruby-lang.org", 443) # port を指定<br>
http.use_ssl = true # use_ssl を true に<br>
http.verify_mode = OpenSSL::SSL::VERIFY_PEER # デフォルトだと検証してくれないので変更<br>
http.start{|h| } # やっと本題<br>
http.finish # 破棄</p>
<p>などとする必要がありました。</p>
<p>この提案では、Net::HTTP(addr, opt){..} という呼び出し方を追加します。<br>
opt はハッシュで、アクセサのある項目を設定する事が出来ます。</p>
<p>なお、折角の新 API なので、この API を使った場合、verify_mode のデフォルトが VERIFY_PEER になっています。<br>
つまり、この API を用いた場合デフォルトで SSL 署名の有効性を検査するため、<br>
期限切れやオレオレ証明書の場合例外が出ます。</p>
<p>これにより、さっきのは以下のように書けます。<br>
Net::HTTP.start("www.ruby-lang.org", use_ssl: true){|h| }</p>
<p>P.S.<br>
なお、以上の例で用いている <a href="http://www.ruby-lang.org" class="external">www.ruby-lang.org</a> は、現在 SSL 証明書が期限切れのため、<br>
この例を今実行すると検証に失敗して例外が出ます。<br>
今は StartCom とかだと無料で 主要環境でルート証明書が入っている Web サーバ用の SSL 証明書が取得できますから、<br>
そういうのに入れ替えた方がいいのではないでしょうか。<br>
=end</p>
Ruby master - Feature #2574 (Closed): merging net/https
https://redmine.ruby-lang.org/issues/2574
2010-01-08T09:40:06Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
現在 net/https はほとんど抜け殻で、ほぼ require 'openssl' するためだけに存在するのですが、<br>
残っているロジックも net/http に移してしまって、net/http だけで https も扱えるようにしませんか。<br>
autoload を使って https に実際にアクセスしたときに openssl ライブラリをロードするようにしたので、<br>
無駄に読み込むこともありません。</p>
<p>なお、net/https は require を使うままにしているので、こちらだともし openssl ライブラリがない場合、<br>
net/https ロード時にエラーが出るという現在の挙動をそのまま利用できます。<br>
=end</p>
Backport191 - Backport #2477 (Closed): String#split should be ASCII sensitive
https://redmine.ruby-lang.org/issues/2477
2009-12-14T15:51:40Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
If r24544 is backported, r24934 is also worth backported.</p>
<p>Author: naruse <a href="mailto:naruse@b2dd03c8-39d4-4d8f-98ff-823fe69b080e" class="email">naruse@b2dd03c8-39d4-4d8f-98ff-823fe69b080e</a><br>
Date: Tue Sep 15 05:27:29 2009 +0000</p>
<pre><code> Use rb_isspace for ASCII-incompatible strings.
* string.c (rb_str_split_m): use rb_isspace when the string
may be ASCII-incompatible.
(rb_str_lstrip_bang): ditto.
(rb_str_rstrip_bang): ditto.
</code></pre>
<p>=end</p>
Ruby master - Feature #2340 (Rejected): Removing YAML/Syck
https://redmine.ruby-lang.org/issues/2340
2009-11-06T17:16:10Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
YAML and Syck is a _why's product and widely used bundled library of Ruby.<br>
But they are not maintained for 2 years and no more by _why.<br>
And they support only YAML 1.0, not 1.1 and 1.2.<br>
So YAML/Syck considered harmful.<br>
=end</p>
Ruby master - Feature #1951 (Closed): openのBOM指定拡張
https://redmine.ruby-lang.org/issues/1951
2009-08-18T23:47:15Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>#747と#802で議論された、openのBOM指定拡張ですが、現在の仕様は、</p>
<ul>
<li>BOMを捨てる</li>
<li>BOMを見てencodingを設定する<br>
という2つの機能が混在しています。</li>
</ul>
<p>このために、たとえば「<code>UTF-8-BOM</code>」という指定でも、<br>
BOMがUTF-16LEを示していた場合には実際に返ってくるStringはUTF-16LEになってしまいます。</p>
<p>この問題に対する解決案として、</p>
<ul>
<li>UTF-*-BOM はBOMを捨てるだけ。別のencodingだった場合は例外</li>
<li>
<code>BOM|UTF-*</code>を追加、これが現在の<code>UTF-*-BOM</code>相当の動作 (BOMを見る OR <code>UTF-*</code>と指定、というイメージ)<br>
というものを考えています。</li>
</ul>
<p>皆さんはどのように思われますか?</p>
Backport191 - Backport #1938 (Closed): FreeBSDでmakeに失敗する
https://redmine.ruby-lang.org/issues/1938
2009-08-14T16:56:45Z
naruse (Yui NARUSE)
naruse@airemix.jp
<p>=begin<br>
common.mkに$(YACC) -d $(YFLAGS) -o y.tab.c $(<:=/)という行がありまして、<br>
これはLinuxとかWindowsでは動くんですが、BSD makeだとバックスラッシュがエスケープだと解釈され、<br>
「Unclosed substitution for < (= missing)」<br>
というエラーが出ます。</p>
<p>この問題はr22964で対処されています。<br>
=end</p>