https://redmine.ruby-lang.org/https://redmine.ruby-lang.org/favicon.ico?17113305112021-11-15T00:15:01ZRuby Issue Tracking SystemRuby master - Feature #18336: How to deal with Trojan Source vulnerabilityhttps://redmine.ruby-lang.org/issues/18336?journal_id=946412021-11-15T00:15:01Zduerst (Martin Dürst)duerst@it.aoyama.ac.jp
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-2 priority-4 priority-default" href="/issues/18337">Bug #18337</a>: Ruby allows zero-width characters in identifiers</i> added</li></ul> Ruby master - Feature #18336: How to deal with Trojan Source vulnerabilityhttps://redmine.ruby-lang.org/issues/18336?journal_id=946522021-11-15T02:50:24Zduerst (Martin Dürst)duerst@it.aoyama.ac.jp
<ul></ul><p>VSCode deal with the Bidi control characters at <a href="https://code.visualstudio.com/updates/v1_62#_unicode-directional-formatting-characters" class="external">https://code.visualstudio.com/updates/v1_62#_unicode-directional-formatting-characters</a>.</p> Ruby master - Feature #18336: How to deal with Trojan Source vulnerabilityhttps://redmine.ruby-lang.org/issues/18336?journal_id=946552021-11-15T05:33:39Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul></ul><p>I'm afraid if prohibiting or warning bidi charaters may bother programmers who use Arabic and/or Hebrew.</p>
<p>Just FYI: Rubocop has an issue to address this issue. <a href="https://github.com/rubocop/rubocop/issues/10226" class="external">https://github.com/rubocop/rubocop/issues/10226</a></p> Ruby master - Feature #18336: How to deal with Trojan Source vulnerabilityhttps://redmine.ruby-lang.org/issues/18336?journal_id=946602021-11-15T10:19:00Zduerst (Martin Dürst)duerst@it.aoyama.ac.jp
<ul></ul><p>mame (Yusuke Endoh) wrote in <a href="#note-3">#note-3</a>:</p>
<blockquote>
<p>I'm afraid if prohibiting or warning bidi charaters may bother programmers who use Arabic and/or Hebrew.</p>
</blockquote>
<p>This is a very good point. I'm not an actual user of the Arabic or Hebrew script (or some other RTL (right-to-left) script such as Syriac,...), but I have done standards work and research in this area, so I'll answer from this perspective.</p>
<p>First, the bidi control characters are not needed to just write a comment in any of these scripts. A comment purely in Arabic or a string purely in Hebrew will automatically be displayed RTL. Bidi controls may be needed if the comment also contains LTR (left-to-right) characters (e.g. Latin or Kanji/Kana), but still can mostly be avoided.</p>
<p>Bidi embeddings (U+202A/U+202B) are only needed if you have a structure of LTR inside RTL inside LTR, but that shouldn't be needed for most comments, and if it looks like it may be needed, it should be possible to avoid this by using more than one line.</p>
<p>Bidi overrides (U+202D/U+202E) are only needed for fixed hard ordering, with the main application I have heard of being part numbers that may contain characters from various scripts. They may also be helpful e.g. if somebody wants to nail down the exact visual output expected.</p>
<p>Bidi isolates (U+2066/U+2067/U+2068) are a relatively new addition. Their main use is as replacement for bidi embeddings, or to isolate pieces with a fixed internal order from outside texts. Their typical use is e.g. when adding items from a database into text in an user interface. So I expect them to appear quite a bit in string interpolations, but in that context, having them escaped would probably help the programmer.</p>
<p>Also, a big advantage is that program text does not get reflowed. Bidi controls are much more important for reflowed text (e.g. in documents or web pages); for texts with fixed linebreaks, some "cheating" is possible (just put pieces of text on the line so they show up the way you want).</p>
<p>On the other hand, programming includes a lot of symbol characters. Many if not most of these symbol characters are 'weak' in the bidi algorithm, i.e. they take their directionality from the surrounding alphabetic characters. But in programming languages, these symbol characters are actually the characters that determine the overall syntax.</p>
<p>This may lead to problems in comments that could be addressed by using bidi controls. More often, it may lead to problems if RTL text is used e.g. for variable names. As an example, the following Ruby fragment has one comment and one variable name in Arabic. The comment looks fine, the assignment of 20 to the variable كتب will need some time to get used to. But that's not the problem in this issue.</p>
<pre><code class="ruby syntaxhl" data-language="ruby"><span class="n">book</span> <span class="o">=</span> <span class="mi">20</span> <span class="c1"># كتب</span>
<span class="err">كتب</span> <span class="o">=</span> <span class="mi">20</span>
</code></pre>
<p>The original paper says that if the bidi control characters are nicely grouped (i.e. each of the opening characters mentioned above is followed by the respective closing character (either PDF, U+202C, or PDI, U+2069), then there is no vulnerability. So it may be possible for comments and strings without interpolations to check for that condition. But I would first like to verify this claim; bidi can be quite tricky.</p> Ruby master - Feature #18336: How to deal with Trojan Source vulnerabilityhttps://redmine.ruby-lang.org/issues/18336?journal_id=948012021-11-22T02:55:36Zduerst (Martin Dürst)duerst@it.aoyama.ac.jp
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Feedback</i></li></ul><p>We discussed this at the developers' meeting on 2021/11/18. No final decision was taken. We think that this issue should primarily by addressed by editors and similar tools, by making the relevant characters visible.</p>
<p>We will see what other languages do; currently, the picture is mixed, with a tendency to leave it to editors,... The only language that we know of that has reacted is Rust. Any feedback is appreciated.</p> Ruby master - Feature #18336: How to deal with Trojan Source vulnerabilityhttps://redmine.ruby-lang.org/issues/18336?journal_id=948432021-11-23T20:39:34ZDan0042 (Daniel DeLorme)
<ul></ul><p>In a sense it's true this is the responsability of the editor, but I also think it's ok to have defense in depth. I would support some form of customizable blacklist of "dangerous" unicode characters that are not allowed in source code, with some sane default.</p>
<pre><code>$UNICODE_BLACKLIST #=> #<Set: {0x202D, 0x202E}> #by default blacklist bidi overrides?
$UNICODE_BLACKLIST << 0x3164 #worried about that "invisible variable" exploit
$UNICODE_BLACKLIST.delete(0x202D).delete(0x202E) #if you need bidi overrides
</code></pre>