Project

General

Profile

Actions

Bug #3765

closed

Ripper::Lexer missed out tokens after '=>' operator

Added by ngtzeyang (TzeYang Ng) about 14 years ago. Updated over 13 years ago.

Status:
Rejected
Assignee:
-
Target version:
-
ruby -v:
ruby 1.9.1p376 (2009-12-07 revision 26041) [x86_64-linux]
Backport:
[ruby-core:31941]

Description

=begin
Given the following ripper use case:

pp Ripper.lex(':x => 1')

I'm getting:

[[[1, 0], :on_symbeg, ":"],
[[1, 1], :on_ident, "x"],
[[1, 2], :on_sp, " "],
[[1, 3], :on_op, "=>"]]

I'm thinking the output should be:

[[[1, 0], :on_symbeg, ":"],
[[1, 1], :on_ident, "x"],
[[1, 2], :on_sp, " "],
[[1, 3], :on_op, "=>"],
[[1, 4], :on_sp, " "],
[[1, 5], :on_int, "1"]]

Somehow the trailing ' 1' is missed out. Seems like a bug ?
=end

Actions #1

Updated by eitoball (Eito Katagiri) about 14 years ago

=begin
I think that it is because the provided statement was illegal and Ripper stopped parsing it.

% ruby -ve ":x => 1"
ruby 1.9.1p378 (2010-01-10 revision 26273) [i386-darwin9.8.0]
-e:1: syntax error, unexpected tASSOC, expecting $end
:x => 1
^

=end

Actions #2

Updated by mame (Yusuke Endoh) about 14 years ago

  • Status changed from Open to Rejected

=begin
Eito is right.

Ruby lexer has very complex states.
Such an incomplete code fragment may not parse.
You should write complete code:

$ ruby -rripper -e 'p *Ripper.lex("foo :x => 1")'
[[1, 0], :on_ident, "foo"]
[[1, 3], :on_sp, " "]
[[1, 4], :on_symbeg, ":"]
[[1, 5], :on_ident, "x"]
[[1, 6], :on_sp, " "]
[[1, 7], :on_op, "=>"]
[[1, 9], :on_sp, " "]
[[1, 10], :on_int, "1"]

--
Yusuke Endoh
=end

Actions

Also available in: Atom PDF

Like0
Like0Like0