Bug #12597
closedProduce test failure if some tests cannot be executed due to lacking data
Description
In some cases, some files (such as Unicode data files) may be missing and cannot be downloaded.
In these cases, tests should not fail silently. It is not good to produce a long list of test failures when there is only a single problem. Therefore, e.g. test/ruby/enc/test_case_comprehensive.rb up to r55520 produced a single explicit failure when tests were missing.
In revision r55521, this test was removed, but it should be reinstated to avoid silent failure.
Updated by jeremyevans0 (Jeremy Evans) over 2 years ago
- Status changed from Open to Closed
Currently, there is a skip instead of a failure if the unicode data files are not available. I tried changing this from a skip to a failure, but it resulted in the CI jobs that run tests failing: https://github.com/jeremyevans/ruby/runs/2859985432#step:15:120
Since CI does not run these tests due to the absence of the unicode data, I don't think it is reasonable that the tests fail if the unicode data is not available. So I'm going to close this. @duerst (Martin Dürst) If you think this is still a bug, please reopen and explain how you think it should work. One possible solution is skipping if we detect that we are running on CI, and failing otherwise, but I'm against that. Another solution is making all CI test jobs download the unicode data files, but I'm guessing that is disabled due to bandwidth issues it would introduce.