Please backport r33937 (newline decorator)
Quoting Yusuke Endoh:
"Yes, text mode is still 10x -- 30x slower than binary mode.
It is reproduced not only on windows but also Linux.
Perhaps, this is the symptom because of the reason explained
And quoting Usaku Nakamura:
"Currently, we has implemented the newline conversion as a
transcode converter, just like encoding conversion.
But the design of transcode is too general to use it such
a simple operation, as our finding.
We want to find a better mechanism which doesn't deviate
from the current design of IO..."
The patch provided by Hiroshi Shirosaki speeds up both file read and write operations under Windows to decent levels while do not deviate from current IO design.
Please consider this for backporting as we see it as Bug and not Feature.
Updated by jonforums (Jon Forums) over 10 years ago
FYI, in the hope that it helps with backporting, a branch containing Hiroshi's patch is being maintained at
The branch also contains backports of r33200 and r33662 so that the following tests pass when building with the RubyInstaller build recipes on Win7 32bit SP1:
make test-all TESTS='openssl fiddle psych zlib io ruby/test_io.rb ruby/test_io_m17n.rb ruby/test_file.rb'
Updated by h.shirosaki (Hiroshi Shirosaki) over 10 years ago
At [ruby-core:42102], Luis mentioned
I took the assignation of above tickets and will perform the backports
next week tops.
If another committer disagree, please say so.
It seems no one disagree.
So I'll do the backports. I heard Luis was very busy with other work.
Backport patches are maintained at
These issues (11 commits) are a set of fixes. So I'll backport with one commit.
I attached a backport patch and commit log.
I'll commit that with version.h PATCHLEVEL fix.
-#define RUBY_PATCHLEVEL 31
+#define RUBY_PATCHLEVEL 32
I confirmed make,
make test and
make test-all are no problem.
Is this OK?
I cannot change redmine issue's status. Could anyone register me as a developer on redmine?
Updated by naruse (Yui NARUSE) over 10 years ago
- Status changed from Assigned to Closed
- % Done changed from 0 to 100
This issue was solved with changeset r34543.
Luis, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.
merge revision(s) 33937: [Backport #5704]
* ext/zlib/zlib.c (rb_gzreader_initialize): use binary mode by default under Windows. Patch by Hiroshi Shirosaki. [ruby-core:40706] [Feature #5562] * include/ruby/encoding.h (void rb_econv_binmode): define NEWLINE decorator. * io.c (rb_cloexec_fcntl_dupfd): Introduce NEED_READCONV and NEED_WRITECONV to replace universal newline decorator by CRLF only when required to improve file reading and writing under Windows. Patch by Hiroshi Shirosaki. [ruby-core:40706] [Feature #5562] * io.c (do_writeconv): adjust binary mode if required. * io.c (read_all, appendline, swallow, rb_io_getline_1): ditto. * io.c (io_getc, rb_io_each_codepoint, rb_io_ungetc): ditto. * io.c (rb_io_binmode, rb_io_ascii8bit_binmode): ditto. * io.c (rb_io_extract_modeenc, rb_sysopen): ditto. * io.c (pipe_open, prep_stdio, io_encoding_set): ditto. * io.c (rb_io_s_pipe, copy_stream_body): ditto. * test/ruby/test_io_m17n.rb (EOT): add test for pipe and stdin in binary mode. * win32/win32.c (init_stdhandle): remove O_BINARY from stdhandle initialization. * win32/win32.c (rb_w32_write): use FTEXT mode accordingly.