https://redmine.ruby-lang.org/https://redmine.ruby-lang.org/favicon.ico?17113305112019-07-04T12:54:34ZRuby Issue Tracking SystemRuby master - Bug #15986: `TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hlhttps://redmine.ruby-lang.org/issues/15986?journal_id=791072019-07-04T12:54:34Zk0kubun (Takashi Kokubun)takashikkbn@gmail.com
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Assigned</i></li><li><strong>Assignee</strong> set to <i>k0kubun (Takashi Kokubun)</i></li></ul><p>Thanks to report. I'd like to know more about the context to fix the issue.</p>
<ul>
<li>Does the error happen at the same place when you retry running the tests?</li>
<li>If so, could you share the output of the following command and all .c/.h files referenced in it?</li>
</ul>
<pre><code>$ ruby --disable-gems --jit-verbose=2 --jit-save-temps --jit-wait --jit-min-calls=2 -e "
def multiply(a, b)
a *= b
end
3.times do
p multiply(7.0, 10.0)
end
"
</code></pre>
<p>In my case, they were <code>/home/k0kubun/.rbenv/versions/ruby/include/ruby-2.7.0/x86_64-linux/rb_mjit_min_header-2.7.0.h</code>, <code>/tmp/_ruby_mjit_p17484u0.c</code>, and <code>/tmp/_ruby_mjit_p17484u1.c</code>. Also the output of <code>ls -la /tmp</code> after that may be also helpful.</p> Ruby master - Bug #15986: `TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hlhttps://redmine.ruby-lang.org/issues/15986?journal_id=792402019-07-09T12:48:54Zvo.x (Vit Ondruch)v.ondruch@tiscali.cz
<ul></ul><p>I wish this was easier to debug. The problem is that this is test failure and it happens on build system, where I don't have access. Trying to reproduce it on my system, this does not work:</p>
<pre><code>$ echo "
def multiply(a, b)
a *= b
end
3.times do
p multiply(7.0, 10.0)
end
" > test.rb
$ make runruby TESTRUN_SCRIPT="--disable-gems --jit-verbose=2 --jit-save-temps --jit-wait --jit-min-calls=2 test.rb"
./revision.h unchanged
./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems --disable-gems --jit-verbose=2 --jit-save-temps --jit-wait --jit-min-calls=2 test.rb
MJIT: CC defaults to /usr/bin/gcc
MJIT: tmp_dir is /tmp
Cannot access header file: /usr/include/rb_mjit_min_header-2.7.0.h
Failure in MJIT header file name initialization
70.0
70.0
70.0
</code></pre>
<p>I have to probably patch the test case to provide me with the output of the files :/</p> Ruby master - Bug #15986: `TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hlhttps://redmine.ruby-lang.org/issues/15986?journal_id=792452019-07-09T14:40:54Zk0kubun (Takashi Kokubun)takashikkbn@gmail.com
<ul><li><strong>Status</strong> changed from <i>Assigned</i> to <i>Feedback</i></li></ul><p>I see. Thanks for the information. At this moment I cannot do anything either, so I'll wait for you to collect the information from the CI system somehow.</p> Ruby master - Bug #15986: `TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hlhttps://redmine.ruby-lang.org/issues/15986?journal_id=792492019-07-09T15:54:39Zmame (Yusuke Endoh)mame@ruby-lang.org
<ul></ul><p>FYI: RubyCI platforms include <a href="https://rubyci.org/logs/rubyci.s3.amazonaws.com/rhel_zlinux/ruby-master/log/20190709T133302Z.log.html.gz#cpu-info" class="external">RHEL 7.1 s390x</a> and <a href="https://rubyci.org/logs/rubyci.s3.amazonaws.com/scw-ad7f67/ruby-master/log/20190709T085405Z.log.html.gz#uname" class="external">Ubuntu armv8 (aarch64)</a>, and their results are both green at the present time. So the cause would be the other factor than CPU, I guess.</p> Ruby master - Bug #15986: `TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hlhttps://redmine.ruby-lang.org/issues/15986?journal_id=792652019-07-10T17:03:04Zvo.x (Vit Ondruch)v.ondruch@tiscali.cz
<ul><li><strong>File</strong> <a href="/attachments/7892">build-s390x.log</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/7892/build-s390x.log">build-s390x.log</a> added</li><li><strong>File</strong> <a href="/attachments/7893">build-armv7hl.log</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/7893/build-armv7hl.log">build-armv7hl.log</a> added</li><li><strong>File</strong> <a href="/attachments/7894">build-x86_64.log</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/7894/build-x86_64.log">build-x86_64.log</a> added</li><li><strong>Subject</strong> changed from <i>`TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and aarch64</i> to <i>`TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hl</i></li></ul><p>So this is my hacked up test case:</p>
<pre><code>$ git diff
diff --git a/test/ruby/test_jit.rb b/test/ruby/test_jit.rb
index 08494cbbbb..9ace7754d4 100644
--- a/test/ruby/test_jit.rb
+++ b/test/ruby/test_jit.rb
@@ -944,9 +944,15 @@ def assert_compile_once(script, result_inspect:, insns: [])
end
# Shorthand for normal test cases
- def assert_eval_with_jit(script, stdout: nil, success_count:, min_calls: 1, insns: [], uplevel: 3)
- out, err = eval_with_jit(script, verbose: 1, min_calls: min_calls)
+ def assert_eval_with_jit(script, stdout: nil, success_count:, min_calls: 2, insns: [], uplevel: 3)
+ out, err = eval_with_jit(script, verbose: 2, min_calls: min_calls, save_temps: true)
actual = err.scan(/^#{JIT_SUCCESS_PREFIX}:/).size
+ puts "", "**********", "* rb_mjit_min_header-2.7.0.h", "---", ""
+ $stdout.flush
+ puts File.read(".ext/include/x86_64-linux/rb_mjit_min_header-2.7.0.h")
+ # puts File.read(".ext/include/armv7hl-linux/rb_mjit_min_header-2.7.0.h")
+ # puts File.read(".ext/include/s390x-linux/rb_mjit_min_header-2.7.0.h")
+ Dir.glob('/tmp/*.c').each {|f| puts '**********', "* #{f}", "", File.read(f), "---"; $stdout.flush}
# Add --jit-verbose=2 logs for cl.exe because compiler's error message is suppressed
# for cl.exe with --jit-verbose=1. See `start_process` in mjit_worker.c.
if RUBY_PLATFORM.match?(/mswin/) && success_count != actual
</code></pre>
<p>And I run just the single test:</p>
<pre><code>make test-all TESTS="test/ruby/test_jit.rb -n /test_block_handler_with_possible_frame_omitted_inlining/"
</code></pre>
<p>See the attached logs from s390x, armv7hl and x86_64 (apologies, some of the lines might be slightly intermingled but the build system, but I hope you can handle that).</p>
<p>BTW I was wrong saying that it fails on AArch64, because it actually fails on armv7hl</p> Ruby master - Bug #15986: `TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hlhttps://redmine.ruby-lang.org/issues/15986?journal_id=793522019-07-12T13:09:17Zk0kubun (Takashi Kokubun)takashikkbn@gmail.com
<ul><li><strong>File</strong> <a href="/attachments/7895">mjit_debug.diff</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/7895/mjit_debug.diff">mjit_debug.diff</a> added</li></ul><p>Thank you. All of the information help me a lot.</p>
<p>It seems that the command line construction is broken for the second compilation in build-armv7hl.log and build-s390x.log, while build-x86_64 seems okay. In this ticket, I attached "mjit_debug.diff" to collect more information on your build environments again. Could you share the build logs with it?</p> Ruby master - Bug #15986: `TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hlhttps://redmine.ruby-lang.org/issues/15986?journal_id=793602019-07-12T15:04:37Zvo.x (Vit Ondruch)v.ondruch@tiscali.cz
<ul><li><strong>File</strong> <a href="/attachments/7896">build-x86_64.log</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/7896/build-x86_64.log">build-x86_64.log</a> added</li><li><strong>File</strong> <a href="/attachments/7897">build-s390x.log</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/7897/build-s390x.log">build-s390x.log</a> added</li><li><strong>File</strong> <a href="/attachments/7898">build-armv7hl.log</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/7898/build-armv7hl.log">build-armv7hl.log</a> added</li></ul><p>Here are the logs (bit messy again, but I hope you can get the information).</p> Ruby master - Bug #15986: `TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hlhttps://redmine.ruby-lang.org/issues/15986?journal_id=793632019-07-12T15:08:08Zvo.x (Vit Ondruch)v.ondruch@tiscali.cz
<ul></ul><p>BTW a bit OT, but seeing all the information stored in the rb_mjit_min_header-2.7.0.h, I am not sure the JIT will work for binary distributions such as Fedora/RHEL. There appears to be embedded a lot of information about the machine used for build, while the JIT has to run on quite different machine.</p> Ruby master - Bug #15986: `TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hlhttps://redmine.ruby-lang.org/issues/15986?journal_id=793642019-07-12T15:51:00Zk0kubun (Takashi Kokubun)takashikkbn@gmail.com
<ul><li><strong>File</strong> <a href="/attachments/7899">mjit_debug2.diff</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/7899/mjit_debug2.diff">mjit_debug2.diff</a> added</li></ul><p>Thank you for the next information. Could you also test the new "mjit_debug2.diff" which I attached now in the same way?</p>
<blockquote>
<p>BTW a bit OT, but seeing all the information stored in the rb_mjit_min_header-2.7.0.h, I am not sure the JIT will work for binary distributions such as Fedora/RHEL.</p>
</blockquote>
<p>MJIT's support policy is that the compiler for runtime MJIT compilation and its path must be the same as one used to build Ruby binary. Otherwise it's just out of support. Even if it's a binary distribution, you could also distribute a compiler as needed.</p> Ruby master - Bug #15986: `TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hlhttps://redmine.ruby-lang.org/issues/15986?journal_id=793982019-07-14T12:49:21Zk0kubun (Takashi Kokubun)takashikkbn@gmail.com
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>Closed</i></li></ul><p>Applied in changeset <a class="changeset" title="Fix a wrong buffer size to avoid stack corruption [Bug #15986]" href="https://redmine.ruby-lang.org/projects/ruby-master/repository/git/revisions/d8cc41c43be65dd4b17e7a6e38f5a7fdf2b247d6">git|d8cc41c43be65dd4b17e7a6e38f5a7fdf2b247d6</a>.</p>
<hr>
<p>Fix a wrong buffer size to avoid stack corruption</p>
<p>[Bug <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: `TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hl (Closed)" href="https://redmine.ruby-lang.org/issues/15986">#15986</a>]</p> Ruby master - Bug #15986: `TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hlhttps://redmine.ruby-lang.org/issues/15986?journal_id=793992019-07-14T12:51:38Zk0kubun (Takashi Kokubun)takashikkbn@gmail.com
<ul></ul><p>Fortunately a very similar issue was reproductive on my macOS machine. I did the mjit_debug2.diff investigation on my own, and noticed the issue fixed by <a class="changeset" title="Fix a wrong buffer size to avoid stack corruption [Bug #15986]" href="https://redmine.ruby-lang.org/projects/ruby-master/repository/git/revisions/d8cc41c43be65dd4b17e7a6e38f5a7fdf2b247d6">d8cc41c43be65dd4b17e7a6e38f5a7fdf2b247d6</a>. And the commit fixed the behavior on my machine. So I hope it's fixed on your environment too.</p> Ruby master - Bug #15986: `TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hlhttps://redmine.ruby-lang.org/issues/15986?journal_id=794442019-07-15T13:14:26Zvo.x (Vit Ondruch)v.ondruch@tiscali.cz
<ul></ul><p>Here are the logs again. I am going to try the latest master and will report back if that helps.</p> Ruby master - Bug #15986: `TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hlhttps://redmine.ruby-lang.org/issues/15986?journal_id=794452019-07-15T13:14:45Zvo.x (Vit Ondruch)v.ondruch@tiscali.cz
<ul><li><strong>File</strong> <a href="/attachments/7902">build-armv7hl.log</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/7902/build-armv7hl.log">build-armv7hl.log</a> added</li><li><strong>File</strong> <a href="/attachments/7903">build-s390x.log</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/7903/build-s390x.log">build-s390x.log</a> added</li><li><strong>File</strong> <a href="/attachments/7904">build-x86_64.log</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/7904/build-x86_64.log">build-x86_64.log</a> added</li></ul> Ruby master - Bug #15986: `TestJIT#test_block_handler_with_possible_frame_omitted_inlining` fails on s390x and armv7hlhttps://redmine.ruby-lang.org/issues/15986?journal_id=796772019-07-16T05:34:40Zvo.x (Vit Ondruch)v.ondruch@tiscali.cz
<ul></ul><p>I did several build with 0c6c937904 and all passed. Thx for the fix.</p>