Feature #22212: Add Thread::Backtrace::Location#source_range

• Status: Open
• Assignee: -
• Target version: -

[Eregon (Benoit Daloze)]

Motivation

The main motivation is to be able to implement Prism.find(Thread::Backtrace::Location) precisely and cleanly on any Ruby implementation,
in a way which does not depend on implementation details like node_id.
For that we need the start/end line/column and the absolute_path, which is exactly what Ruby::SourceRange provides.

In #21998 we added source_range for {Method,UnboundMethod,Proc}.

However we also need source_range for Thread::Backtrace::Location as this is used in error_highlight, power_assert, etc.
Those tools currently use RubyVM::AbstractSyntaxTree.node_id_for_backtrace_location(Thread::Backtrace::Location), however:

• that's a CRuby-only experimental API
• it exposes CRuby internals (node_id)
• it adds a lot of complexity for all usages because they need to handle both Prism::Node and RubyVM::AbstractSyntaxTree::Node.

We solve all of this by adding a new portable API which exposes universal concepts like line and column, which are stable to find AST nodes.

I also know this API can be implemented on other Ruby implementations, notably on TruffleRuby without needing node_id, which illustrates its generality and portability.

Implementation

How can we get the start/end line/column when we only keep the start line in CRuby?
By re-parsing, and specifically by re-parsing with exactly the same parser that the interpreter used to compile to bytecode.
That way there is no issue of using different Prism versions, it's the Prism C library built in CRuby, or parse.y in --parser=parse.y mode.
Either way, we reparse to extract the start/end line/column.
We make the returned start/end line/column match for both Prism and parse.y, in practice the only difference is for calls with blocks, where we adapt the location from parse.y to match Prism.
(the fact the lines & columns match for both parsers for all cases except blocks also illustrates the stability of those locations)

One can then easily use Prism to get a node matching that Ruby::SourceRange and use the resulting Prism::Node as they wish.
This also means all versions of Prism can be used, and for example if some dependency requires some specific version of Prism it works fine and is not a problem.
As a result, the user of this API can choose the Prism version they want and always get a Prism::Node (and not having to handle RubyVM::AbstractSyntaxTree::Node too, or a duplicate Ruby::Node API as mentioned in #21795).

PR: https://github.com/ruby/ruby/pull/18043

I also made a PR to update ErrorHighlight to use this new API and prove how well it works: https://github.com/eregon/ruby/pull/2
This in turn will be a major step towards getting ErrorHighlight to work on other implementations than CRuby.

[Eregon (Benoit Daloze)]

• Related to Feature #21795: Methods for retrieving ASTs added

[Eregon (Benoit Daloze)]

• Related to Feature #21998: Add {Method,UnboundMethod,Proc}#source_range added

[Eregon (Benoit Daloze)]

• Description updated (diff)

[Eregon (Benoit Daloze)]

• Description updated (diff)

[Eregon (Benoit Daloze)]

• Description updated (diff)

[Eregon (Benoit Daloze)]

• Description updated (diff)

[mame (Yusuke Endoh)]

Why not just use Backtrace::Location#syntax_tree?

As the maintainer of error_highlight, I have no plan to use this method. I will use Backtrace::Location#syntax_tree. Therefore, the use cases for this feature should be explained separately.