Project

General

Profile

Actions

Bug #19291

closed

Ripper's on_parse_error token has wrong content

Added by tompng (tomoya ishida) almost 2 years ago. Updated almost 2 years ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-darwin20]
[ruby-core:111556]

Description

Ruby 3.1

Ripper::Lexer.new("{a:,aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\nhelloworld").scan.map(&:tok)
# =>
["{", "a:", ",", "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "", "\n", "helloworld"]

Ruby 3.2.0

Ripper::Lexer.new("{a:,aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\nhelloworld").scan.map(&:tok)
# => 
["{",
 "a:",
 ",",
 "oworld\u0000\u0000\u0000\u0000\u0000\u0000z`\u0001\u0000\u0000\u0000\u0000\u0000\xC0\xF0$\u0010\u0001\u0000\u0000\u0000\x80A\xB8\u0019\xB5\u007F\u0000\u0000\x91\x94",
 "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
 "\n",
 "helloworld"]
Actions #1

Updated by nobu (Nobuyoshi Nakada) almost 2 years ago

  • Status changed from Open to Closed

Applied in changeset git|3becc4a105bcf873fdc6e83c2a957f73718c5084.


[Bug #19291] Rewind to the previous line

When rewinding looking ahead after newline token, also reset the last
line string, the pointers to it, and the location, not only the line
number.

Updated by nobu (Nobuyoshi Nakada) almost 2 years ago

This may be a potential bug since older versions.

Actions

Also available in: Atom PDF

Like0
Like0Like0