Bug #3765
closedRipper::Lexer missed out tokens after '=>' operator
Description
=begin
Given the following ripper use case:
pp Ripper.lex(':x => 1')
I'm getting:
[[[1, 0], :on_symbeg, ":"],
[[1, 1], :on_ident, "x"],
[[1, 2], :on_sp, " "],
[[1, 3], :on_op, "=>"]]
I'm thinking the output should be:
[[[1, 0], :on_symbeg, ":"],
[[1, 1], :on_ident, "x"],
[[1, 2], :on_sp, " "],
[[1, 3], :on_op, "=>"],
[[1, 4], :on_sp, " "],
[[1, 5], :on_int, "1"]]
Somehow the trailing ' 1' is missed out. Seems like a bug ?
=end
Updated by eitoball (Eito Katagiri) about 14 years ago
=begin
I think that it is because the provided statement was illegal and Ripper stopped parsing it.
% ruby -ve ":x => 1"
ruby 1.9.1p378 (2010-01-10 revision 26273) [i386-darwin9.8.0]
-e:1: syntax error, unexpected tASSOC, expecting $end
:x => 1
^
=end
Updated by mame (Yusuke Endoh) about 14 years ago
- Status changed from Open to Rejected
=begin
Eito is right.
Ruby lexer has very complex states.
Such an incomplete code fragment may not parse.
You should write complete code:
$ ruby -rripper -e 'p *Ripper.lex("foo :x => 1")'
[[1, 0], :on_ident, "foo"]
[[1, 3], :on_sp, " "]
[[1, 4], :on_symbeg, ":"]
[[1, 5], :on_ident, "x"]
[[1, 6], :on_sp, " "]
[[1, 7], :on_op, "=>"]
[[1, 9], :on_sp, " "]
[[1, 10], :on_int, "1"]
--
Yusuke Endoh mame@tsg.ne.jp
=end