Project

General

Profile

Actions

Bug #5202

closed

RubyLex very slow when lexing large files

Added by ryanmelt (Ryan Melton) about 13 years ago. Updated almost 12 years ago.

Status:
Closed
Target version:
-
ruby -v:
ruby 1.9.2p290 (2011-07-09) [i386-mingw32]
Backport:
[ruby-core:39018]

Description

RubyLex exponentially slower when lexing large files based on the size of the file. This is because of a call to @readed.reverse inside of the get_readed and ungetc methods. The following updated versions of these methods greatly improve performance by using rindex instead of reverse.index.

def get_readed
if idx = @readed.rindex("\n")
@base_char_no = @readed.size - (idx + 1)
else
@base_char_no += @readed.size
end

readed = @readed.join("")
@readed = []
readed

end

def ungetc(c = nil)
if @here_readed.empty?
c2 = @readed.pop
else
c2 = @here_readed.pop
end
c = c2 unless c
@rests.unshift c #c =
@seek -= 1
if c == "\n"
@line_no -= 1
if idx = @readed.rindex("\n")
@char_no = idx + 1
else
@char_no = @base_char_no + @readed.size
end
else
@char_no -= 1
end
end

Updated by naruse (Yui NARUSE) about 13 years ago

  • Status changed from Open to Assigned
  • Assignee set to keiju (Keiju Ishitsuka)
Actions #2

Updated by keiju (Keiju Ishitsuka) almost 12 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r38609.
Ryan, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


  • lib/irb/ruby-lex.rb: improve RubyLex performance for large files
    [Bug #5202]. Patch by ryanmelt.
Actions

Also available in: Atom PDF

Like0
Like0Like0