Project

General

Profile

Bug #13962

Change http://unicode.org to https

Added by MSP-Greg (Greg L) over 2 years ago. Updated about 2 months ago.

Status:
Closed
Priority:
Normal
Target version:
-
ruby -v:
ruby 2.5.0dev (2017-10-01 trunk 60085) [x64-mingw32]
[ruby-core:83074]
Tags:

Description

I believe downloads from unicode.org can be done via https.

See attached patch.

Thank you.


Files

unicode.org.patch (435 Bytes) unicode.org.patch http -> https MSP-Greg (Greg L), 10/01/2017 08:09 PM

Related issues

Related to Ruby master - Misc #13974: Make sure Unicode files are only downloaded once, not repeatedly, for continuous integrationClosedActions
Related to Ruby master - Bug #13918: Appveyor failure - svn 59961 Use https instead of ftp for libffi downloadingAssignedhsbt (Hiroshi SHIBATA)Actions

Updated by duerst (Martin Dürst) over 2 years ago

MSP-Greg (Greg L) wrote:

I believe downloads from unicode.org can be done via https.

Yes, that seems to be the case. Let me check with my contacts at the Unicode Consortium to see what they prefer (in particular for large data downloads).

Updated by shevegen (Robert A. Heiler) over 2 years ago

Secure our emojis! \o/

Updated by MSP-Greg (Greg L) over 2 years ago

shevegen (Robert A. Heiler) wrote:

Secure our emojis! \o/

Yeah, I've lost a few nights' sleep worrying about that...

I've got a patch to tool/downloader.rb that outputs the file size and URI, and I noticed it doing a local build. I think it's just good practice that all downloads are done via https, regardless of the 'threat potential' of the files.

Updated by duerst (Martin Dürst) over 2 years ago

  • Assignee set to duerst (Martin Dürst)

Updated by duerst (Martin Dürst) over 2 years ago

Just an intermediate report: HTTPS is available only since about a week, and the Unicode Consortium wants to check things a bit more before the availability is officially confirmed and announced. I'll wait until that time.

#6

Updated by duerst (Martin Dürst) over 2 years ago

  • Related to Misc #13974: Make sure Unicode files are only downloaded once, not repeatedly, for continuous integration added

Updated by normalperson (Eric Wong) over 2 years ago

duerst@it.aoyama.ac.jp wrote:

Just an intermediate report: HTTPS is available only since
about a week, and the Unicode Consortium wants to check things
a bit more before the availability is officially confirmed and
announced. I'll wait until that time.

Regardless of HTTPS or not; can we keep known-good
SHA-256/384/512/whatever signature(s) of the to-be-downloaded
files in our repository and validate the downloaded result?

IIRC, MiTM HTTPS proxies exist, and the CA system is still
vulnerable.

Updated by duerst (Martin Dürst) over 2 years ago

normalperson (Eric Wong) wrote:

Regardless of HTTPS or not; can we keep known-good
SHA-256/384/512/whatever signature(s) of the to-be-downloaded
files in our repository and validate the downloaded result?

IIRC, MiTM HTTPS proxies exist, and the CA system is still
vulnerable.

Unicode is currently looking at adding checksums. We should definitely integrate these into our process when they are available.

Also, please note that while the Unicode files get downloaded when compiling from scratch, we actually process them and commit the result into our repository (e.g. enc/unicode/10.0.0/casefold.h and enc/unicode/10.0.0/name2ctype.h). So any fishy stuff would quickly be detected if it generated diffs for these files.

#9

Updated by duerst (Martin Dürst) over 2 years ago

  • Status changed from Open to Closed

Updated by hsbt (Hiroshi SHIBATA) over 2 years ago

  • Status changed from Closed to Assigned
#11

Updated by hsbt (Hiroshi SHIBATA) over 2 years ago

  • Related to Bug #13918: Appveyor failure - svn 59961 Use https instead of ftp for libffi downloading added

Updated by znz (Kazuhiro NISHIYAMA) about 2 months ago

Making snapshots of 2.5 and 2.6 sometimes failed to download from http://www.unicode.org.
But it seems making snapshots of 2.7 and master is no error.
So I want to retry this in ruby_2_5 and ruby_2_6.

https://github.com/ruby/actions/runs/576783877?check_suite_focus=true#step:4:21

Failed to open TCP connection to www.unicode.org:80 (Connection timed out - connect(2) for "www.unicode.org" port 80): http://www.unicode.org/Public/10.0.0/ucd/UnicodeData.txt

https://github.com/ruby/actions/runs/576782985?check_suite_focus=true#step:4:20

retrying Errno::ETIMEDOUT (Failed to open TCP connection to www.unicode.org:80 (Connection timed out - connect(2) for "www.unicode.org" port 80)) after 1 seconds...

Updated by duerst (Martin Dürst) about 2 months ago

znz (Kazuhiro NISHIYAMA) wrote in #note-13:

Making snapshots of 2.5 and 2.6 sometimes failed to download from http://www.unicode.org.
But it seems making snapshots of 2.7 and master is no error.
So I want to retry this in ruby_2_5 and ruby_2_6.

This may be related to http://blog.unicode.org/2020/04/technical-alert-unicode-technical.html. I have contacted the Unicode Consortium, and will report back here when I learn more about it.

Because this issue is closed, I suggest opening a new one.

Also available in: Atom PDF