Bug #18636: Problems with version check for file emoji-variation-sequences.txt

• Status: Closed
• Assignee: duerst (Martin Dürst)
• Target version: -
• Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN

[duerst (Martin Dürst)]

[This problem was detected and reported (on Slack) by Koichi Sasada (@ko1 (Koichi Sasada)). Koichi also provided a short-time patch. I'm writing this issue to better document the problem (and the solution) for later reference.]

Example of original failure:
http://ci.rvm.jp/results/trunk-random3@phosphorus-docker/3873286

The direct reason for this failure is that between beta and final, the file changed as follows (< is beta version (my local), > is final version (used in some CI setups, other CI setups don't use these files at all to save time)):

1,2c1,2
< # emoji-variation-sequences.txt
< # Date: 2021-04-08, 18:35:21 GMT
---
> # emoji-variation-sequences-14.0.0.txt
> # Date: 2021-06-08, 05:19:16 GMT
8c8
< # Version: 14.0
---
> # Used with Emoji Version 14.0 and subsequent minor revisions (if any)

This explains why my 'fix' to check for "14.0" (in commit 48f1e8c5d850) didn't work in the relevant CIs.

What is not clear, and may never become clear, is why I didn't detect this problem earlier.
Also, the question is how to make sure the relevant data files get updated for future updates of Unicode.
In commit 6d603cfde1ca, I made sure more all relevant data files get downloaded, but it's not fully clear to me whether this will avoid problems in the future.

The above beta version is identical in style to Unicode 13.0.0/Emoji 13.0 (https://unicode.org/Public/13.0.0/ucd/emoji/emoji-variation-sequences.txt), it reads:

# emoji-variation-sequences.txt
# Date: 2020-01-21, 07:15:05 GMT
# © 2020 Unicode®, Inc.
# Unicode and the Unicode Logo are registered trademarks of Unicode, Inc. in the U.S. and other countries.
# For terms of use, see http://www.unicode.org/terms_of_use.html
#
# Emoji Variation Sequences for UTS #51
# Version: 13.0

I actually found mail that indicates that it was me who complained about this mismatch (Emoji version in file in an Unicode-version numbered (sub-)directory). So this is a typical example of shooting yourself in the foot.

[duerst (Martin Dürst)]

• Blocks Feature #18037: Update Unicode data to Unicode Version 14.0.0 added

[duerst (Martin Dürst)]

• Status changed from Open to Closed

This was fixed with the following commits and can now be closed.

commit 151208a19b09e29d45a79d92d73b67ec2ff692a6
Author: Martin Dürst <[e-mail]>
Date:   Wed Mar 16 15:18:30 2022 +0900

    Revert "Fix version check to use Emoji version for emoji-variation-sequences.txt"

    This reverts commit 48f1e8c5d85043e6adb8e93c94532daa201d42e9.

commit e85547ae81e126a89a7bc9436af4a26c24c6f754
Author: Martin Dürst <[e-mail]>
Date:   Wed Mar 16 15:17:30 2022 +0900

    Revert "Allow `.0` version mismatch to pass the tests"

    This reverts commit fc6e4ce62bfa95b6a0d4d4898e1128c1fce4db8a.

commit 6d603cfde1cadcc44b8b331459832f2675594702
Author: Martin Dürst <[e-mail]>
Date:   Wed Mar 16 15:15:21 2022 +0900

    add Unicode auxiliary files and Unicode UCD emoji files download for ALWAYS_UPDATE_UNICODE