Project

General

Profile

Actions

Bug #20809

open

Ripper fails to parse/tokenize nested heredocs within embexpr

Added by tompng (tomoya ishida) about 1 month ago.

Status:
Open
Assignee:
-
Target version:
-
ruby -v:
ruby 3.4.0dev (2024-10-22T09:04:30Z master 54065f3b7b) +PRISM [x86_64-linux]
[ruby-core:119587]

Description

In this example, Ripper wrongly concats "s333\n" and "s4444".

require 'ripper'

code1 = '<<HEREDOC1
s1
#{<<HEREDOC2}s333
s22
HEREDOC2
s4444#{55555}
HEREDOC1'

p Ripper.tokenize(code1)
# => ["<<HEREDOC1", "\n", "s1\n", "\#{", "<<HEREDOC2", "}", "s333\ns4444", "s22\n", "HEREDOC2\n", "\#{", "55555", "}", "\n", "HEREDOC1"]

In this example, result of Ripper.sexp is wrong. "s333" disappears.

code2 = '<<HEREDOC1
s1
#{<<HEREDOC2}s333
s22
HEREDOC2
s4444
HEREDOC1'

p eval(code2) # => "s1\ns22\ns333\ns4444\n"
p Ripper.sexp(code2)
# => 
[:program,
 [[:string_literal,
   [:string_content,
    [:@tstring_content, "s1\n", [2, 0]],
    [:string_embexpr,
     [[:string_literal,
       [:string_content,
        [:@tstring_content, "s22\n", [4, 0]]]]]],
    [:@tstring_content, "s4444\n", [6, 0]]]]]]

No data to display

Actions

Also available in: Atom PDF

Like0