Bug #17211: Test failures in ruby2.7.2 and ruby3.0~preview1

• Status: Closed
• Assignee: aycabta (aycabta .)
• Target version: -
• ruby -v: 2.7.2
• Backport: 2.5: DONTNEED, 2.6: DONTNEED, 2.7: DONE

[utkarsh (Utkarsh Gupta)]

Hello,

Whilst building Ruby 2.7.2 and Ruby 3.0~preview1, I got the following test failures:

  1) Failure:
TestIRB::TestHistory#test_history_save_1 [/<<PKGBUILDDIR>>/test/irb/test_history.rb:31]:
<"exit\n"> expected but was
<"1\n" + "2\n" + "3\n" + "4\n">.

  2) Failure:
TestIRB::TestHistory#test_history_save_100 [/<<PKGBUILDDIR>>/test/irb/test_history.rb:51]:
<"1\n" + "2\n" + "3\n" + "4\n" + "5\n" + "exit\n"> expected but was
<"1\n" + "2\n" + "3\n" + "4\n">.

  3) Failure:
TestIRB::TestHistory#test_history_save_bignum [/<<PKGBUILDDIR>>/test/irb/test_history.rb:76]:
<"1\n" + "2\n" + "3\n" + "4\n" + "5\n" + "exit\n"> expected but was
<"1\n" + "2\n" + "3\n" + "4\n">.

  4) Failure:
TestIRB::TestHistory#test_history_save_minus_as_infinity [/<<PKGBUILDDIR>>/test/irb/test_history.rb:101]:
<"1\n" + "2\n" + "3\n" + "4\n" + "5\n" + "exit\n"> expected but was
<"1\n" + "2\n" + "3\n" + "4\n">.

Finished tests in 424.892771s, 49.5113 tests/s, 6415.2304 assertions/s.
21037 tests, 2725785 assertions, 4 failures, 0 errors, 95 skips

I am clueless as to what's causing this?
Please note that I tried to build it in a clean environment. But couldn't get these tests to pass, neither in ruby2.7.2 or ruby3.0~preview1.

[mame (Yusuke Endoh)]

Hi, cloud you tell us what environment (OS, distribution, version, CPU, terminal, etc.) you are using? Thanks

[utkarsh (Utkarsh Gupta)]

Hey,

mame (Yusuke Endoh) wrote in #note-1:

Hi, cloud you tell us what environment (OS, distribution, version, CPU, terminal, etc.) you are using? Thanks

Right, sorry for not mentioning this earlier.
I am running Debian Sid. Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz with 16 gigs of RAM (in case that's of any help).
I co-maintain Ruby in Debian and I wanted to upload v2.7.2 but stumbled upon these failures. However, 2.7.1 just builds fine!
Interestingly, I get the same error with 3.0.0~preview1 as well.

Here are the Debina CI logs: https://salsa.debian.org/ruby-team/ruby/-/jobs/1044445#L8148

Please let me know if you need any more information.

[utkarsh (Utkarsh Gupta)]

utkarsh (Utkarsh Gupta) wrote:

I am clueless as to what's causing this?

Weirdly enough it looks to me that

IRB_HISTORY
  stdin.write("5\nexit\n")
end

in test/irb/test_history.rb (for all these tests) has no impact apparently. But not sure what's going on with test_history_save_1. I am lost there.

[mame (Yusuke Endoh)]

Thanks. I tested make && make check on Debian sid x86_64 on Docker on WSL2, but couldn't reproduce the issue. I'm not familiar with the deb packaging process, so can you explain how to reproduce the issue?

[mame (Yusuke Endoh)]

• Status changed from Open to Assigned
• Assignee set to aycabta (aycabta .)

Okay, I got it. Those tests fail when no executable named ruby is available in PATH.

https://github.com/ruby/ruby/blob/f8c50109d7d1a067bae80b56e7b0b1923951c068/test/irb/test_history.rb#L131

Note that the build directory is added to PATH during the test. So, the problem occurs typically (1) when there is no system ruby (such as /usr/bin/ruby), and (2) when the executable being built is renamed, for example, by ./configure --program-suffix=something.

Sorry for the inconvenience. This is a bug of the tests. I'm assigning @aycabta (aycabta .).

[aycabta (aycabta .)]

I decided to stop using system method for E2E testing because it depends on ruby command. I'm sorry for the rough implementation.

https://github.com/ruby/ruby/pull/3628

This patch of this Pull Request resolves the failure of CI.

[utkarsh (Utkarsh Gupta)]

Hello,

aycabta (aycabta .) wrote in #note-6:

I decided to stop using system method for E2E testing because it depends on ruby command. I'm sorry for the rough implementation.

https://github.com/ruby/ruby/pull/3628

This patch of this Pull Request resolves the failure of CI.

Thank you so much! I can confirm that this indeed fixes the issues on the Debian side! :)

Thanks for all your work! \o/

[jeremyevans0 (Jeremy Evans)]

• Status changed from Assigned to Closed

Fixed by 20ad1017017ea736667d86fa0250dc1a39daefa1.

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN to 2.5: DONTNEED, 2.6: DONTNEED, 2.7: REQUIRED

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.5: DONTNEED, 2.6: DONTNEED, 2.7: REQUIRED to 2.5: DONTNEED, 2.6: DONTNEED, 2.7: DONE

ruby_2_7 d1ba5545513b68d39ca29b578a42bd8d48a7804e merged revision(s) 9718ff62c12c07ecf7f0e234343dca76ee1aa51d,20ad1017017ea736667d86fa0250dc1a39daefa1.