Bug #20343: Ripper.sexp in Ruby 3.4 returns nil

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 3.4.0dev (2024-03-17T10:09:37Z master 5fd6b461c7) [x86_64-darwin23]
• Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN

[koic (Koichi ITO)]

In Ruby 3.4.0's Ripper, there is the following incompatible behavior:

Expected (Ruby 3.3)

It is expected that tokens will be returned.

$ ruby -rripper -ve "p Ripper.sexp('foo[bar, &baz] += 1')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
[:program, [[:opassign, [:aref_field, [:vcall, [:@ident, "foo", [1, 0]]], [:args_add_block, [[:vcall, [:@ident, "bar", [1, 4]]]], [:vcall, [:@ident, "baz", [1, 10]]]]], [:@op, "+=", [1, 15]], [:@int, "1", [1, 18]]]]]

$ ruby -rripper -ve "p Ripper.sexp_raw('foo[bar, &baz] += 1')"
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin22]
[:program, [:stmts_add, [:stmts_new], [:opassign, [:aref_field, [:vcall, [:@ident, "foo", [1, 0]]], [:args_add_block, [:args_add, [:args_new], [:vcall, [:@ident, "bar", [1, 4]]]], [:vcall, [:@ident, "baz", [1, 10]]]]], [:@op, "+=", [1, 15]], [:@int, "1", [1, 18]]]]]

Actual (Ruby 3.4)

nil is returned.

$ ruby -rripper -ve "p Ripper.sexp('foo[bar, &baz] += 1')"
ruby 3.4.0dev (2024-03-17T10:09:37Z master 5fd6b461c7) [x86_64-darwin23]
nil

$ ruby -rripper -ve "p Ripper.sexp_raw('foo[bar, &baz] += 1')"
ruby 3.4.0dev (2024-03-17T10:09:37Z master 5fd6b461c7) [x86_64-darwin23]
nil

The same applies to the following cases.

foo[bar, &baz] += 1
foo[bar, &baz] ||= 1
foo[bar, &baz] &&= 1
foo.foo[bar, &baz] += 1
foo.foo[bar, &baz] ||= 1
foo.foo[bar, &baz] &&= 1

This was discovered in a failure of Prism's CI:
https://github.com/ruby/prism/actions/runs/8317131929/job/22757514162?pr=2607

[jeremyevans0 (Jeremy Evans)]

• Status changed from Open to Closed

This is expected as that syntax was removed. See #19918