Bug #19736: Ripper.tokenize ignores token after heredoc_beg when embexpr inside heredoc is unclosed

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 3.3.0dev (2023-06-19T09:28:10Z master 2a80bac9f0) [x86_64-linux]
• Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN

[tompng (tomoya ishida)]

I expect Ripper.tokenize(code) to tokenize +1+2+3\n part.

code = <<'RUBY'
<<A+1+2+3
#{4
RUBY
Ripper.tokenize code
# => ["<<A", "\#{", "4", "\n"]

It will be easy for IRB's internal code to handle indent and other things if Ripper.tokenize can tokenize all part of the code even if it is incomplete.

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed

Applied in changeset git|6be402e172a537000de58a28af389cb55dd62ec8.

---

[Bug #19736] Recover after unterminated interpolation