Project

General

Profile

Actions
Copy link

Bug #5202

closed

RubyLex very slow when lexing large files

Added by ryanmelt (Ryan Melton) over 13 years ago. Updated almost 12 years ago.

Status:
Closed
Assignee:
keiju (Keiju Ishitsuka)
Target version:
-
ruby -v:
ruby 1.9.2p290 (2011-07-09) [i386-mingw32]
Backport:
[ruby-core:39018]

Description

RubyLex exponentially slower when lexing large files based on the size of the file. This is because of a call to @readed.reverse inside of the get_readed and ungetc methods. The following updated versions of these methods greatly improve performance by using rindex instead of reverse.index.

def get_readed
if idx = @readed.rindex("\n")
@base_char_no = @readed.size - (idx + 1)
else
@base_char_no += @readed.size
end

readed = @readed.join("")
@readed = []
readed

end

def ungetc(c = nil)
if @here_readed.empty?
c2 = @readed.pop
else
c2 = @here_readed.pop
end
c = c2 unless c
@rests.unshift c #c =
@seek -= 1
if c == "\n"
@line_no -= 1
if idx = @readed.rindex("\n")
@char_no = idx + 1
else
@char_no = @base_char_no + @readed.size
end
else
@char_no -= 1
end
end

Actions
Copy link

Also available in: Atom PDF

Like0
Like0Like0