Bug #15924
closedRipper::Lexer RuntimeError - Ripper FATAL - master/trunk
Description
YARD based docs systems use Ripper for highlighting both source code and 'code snippets' in comments and rdoc/md files.
Using the build shown, I have an error as shown:
lib/ruby/2.7.0/ripper/lexer.rb:101:in `lex'
lib/ruby/2.7.0/ripper/lexer.rb:123:in `parse'
lib/ruby/2.7.0/ripper/lexer.rb:123:in `parse': [Ripper FATAL] unknown token -1 (RuntimeError)
A quick look shows that it was parsing the rdoc files in the doc folder. Sorry for not being able to check further tonite, I'll look more tomorrow...
Updated by MSP-Greg (Greg L) almost 5 years ago
Found the issue, below is a repo, which in MinGW master, generates RuntimeError...
require 'ripper'
temp = "<<\"EOS\n\" # This has been warned since 2.4\nEOS"
puts '', temp, ''
lexer = Ripper::Lexer.new temp
lexer.lex.each { |t| pp t }
puts "lexer.error? #{lexer.error?}"
Updated by MSP-Greg (Greg L) almost 5 years ago
- Subject changed from Ripper FATAL - master/trunk to Ripper::Lexer RuntimeError - Ripper FATAL - master/trunk
Updated by nobu (Nobuyoshi Nakada) almost 5 years ago
It is an example of syntax error, cannot be fallen back?
YARD itself just warns it but succeeds.
$ echo $'<<\"EOS\n\"\nEOS' > yardtest.rb
$ ~/.gem/ruby/2.7.0/bin/yard doc yardtest.rb
[warn]: Syntax error in `yardtest.rb`:(1,0): unterminated here document identifier
Files: 0
Modules: 0 ( 0 undocumented)
Classes: 0 ( 0 undocumented)
Constants: 0 ( 0 undocumented)
Attributes: 0 ( 0 undocumented)
Methods: 0 ( 0 undocumented)
100.00% documented
Updated by MSP-Greg (Greg L) almost 5 years ago
Thanks. Actually, YARD doesn't use the Lexer, but I switched to using it a long time ago, and all the repos shown at https://msp-greg.github.io/ are created using it. Conversely, Lexer is subclassed from Ripper. So what's using what is a little vague...
Until recently, all 'bad' code snippets have flipped Ripper#error? to true with no RuntimeError. I guess it is a edge case. Ok to close...
Updated by jeremyevans0 (Jeremy Evans) over 4 years ago
- Status changed from Open to Closed