Project

General

Profile

Actions

Bug #2742

closed

IO#read/gets can be very slow in doze

Added by rogerdpack (Roger Pack) about 14 years ago. Updated almost 13 years ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
ruby 1.9.2dev (2010-02-12 trunk 26649) [i386-mingw32]
Backport:
[ruby-core:28162]

Description

=begin
this code

n = 100 * 1000 * 1000
puts "writing"
File.open("foo", 'wb'){|f| f.write(" " * n) }
puts "reading"
File.open("foo", 'r') do |io|
io.read
end

takes something like 700s in windows, most of the time is spent in rb_str_resize. Cause seems to be that it is rb_str_resize'ing the string by 1024B at a time, so N^2 in the amount of time it takes to read it in because it has to keep copying it to the newly sized string.

The same thing happens for

File.open("foo", 'rb') do |io|
io.gets
end

except that it reads (and resizes) in 8K bytes. Not sure why they're different sizes. I'd imagine this problem exists in Linux, but without as much of a negative impact as it reads in larger sizes.
=end


Related issues 2 (0 open2 closed)

Related to Backport187 - Backport #2741: gets with large file is slow in windozeClosedActions
Related to Ruby master - Bug #1332: Reading file on Windows is 500x slower then with previous Ruby versionClosedakr (Akira Tanaka)03/30/2009Actions
Actions #1

Updated by rogerdpack (Roger Pack) about 14 years ago

=begin
This seems to help, though I'm not sure if it's the best way.

Index: io.c

--- io.c (revision 26655)
+++ io.c (working copy)
@@ -1170,7 +1170,7 @@
if (fptr->rbuf == NULL) {
fptr->rbuf_off = 0;
fptr->rbuf_len = 0;

  •    fptr->rbuf_capa = 8192;
    
  •    fptr->rbuf_capa = 64*1024;
       fptr->rbuf = ALLOC_N(char, fptr->rbuf_capa);
    
    }
    if (fptr->rbuf_len == 0) {
    @@ -1583,7 +1583,7 @@
    rb_exc_raise(rb_econv_open_exc(sname, dname, ecflags));
    fptr->cbuf_off = 0;
    fptr->cbuf_len = 0;
  •    fptr->cbuf_capa = size < 1024 ? 1024 : size;
    
  •    fptr->cbuf_capa = size < (64*1024) ? (64*1024) : size;
       fptr->cbuf = ALLOC_N(char, fptr->cbuf_capa);
    
    }
    }

=end

Actions #2

Updated by nobu (Nobuyoshi Nakada) about 14 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

=begin
This issue was solved with changeset r26997.
Roger, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

=end

Actions

Also available in: Atom PDF

Like0
Like0Like0