https://redmine.ruby-lang.org/https://redmine.ruby-lang.org/favicon.ico?17113305112011-08-28T02:13:36ZRuby Issue Tracking SystemRuby master - Bug #5239: bootstraptest/runner.rb: assert_normal_exit logic broken on Debian/GNU kFreeBSDhttps://redmine.ruby-lang.org/issues/5239?journal_id=204472011-08-28T02:13:36Zlucas (Lucas Nussbaum)lucas@lucas-nussbaum.net
<ul><li><strong>File</strong> <a href="/attachments/2028">assert_normal_exit.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/2028/assert_normal_exit.patch">assert_normal_exit.patch</a> added</li></ul><p>The attached patch fixes this. Could you consider applying to 1.9.3 too?</p>
<p>Thanks</p> Ruby master - Bug #5239: bootstraptest/runner.rb: assert_normal_exit logic broken on Debian/GNU kFreeBSDhttps://redmine.ruby-lang.org/issues/5239?journal_id=204482011-08-28T02:30:03Zkosaki (Motohiro KOSAKI)kosaki.motohiro@gmail.com
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Assigned</i></li><li><strong>Assignee</strong> set to <i>kosaki (Motohiro KOSAKI)</i></li><li><strong>Target version</strong> set to <i>1.9.3</i></li></ul><blockquote>
<p>The problem is that the call to $? in the thread cannot retrieve the exit value of the process started by popen. "$?" is<br>
transformed into a wait4() syscall, but wait4 is not allowed to inquire the state of non-child processes. And the popen<br>
process is not a child of the sub-thread, it is a child of the main thread.</p>
<p><a href="http://pubs.opengroup.org/onlinepubs/009695399/functions/wait.html" class="external">http://pubs.opengroup.org/onlinepubs/009695399/functions/wait.html</a> confirms that waitpid() is not supposed to work on<br>
non-child processes.</p>
</blockquote>
<p>In fact, it's a child process. see more below paragraph.</p>
<blockquote>
<p>RATIONALE<br>
A call to the wait() or waitpid() function only returns status on an immediate child process of the <em>calling process</em></p>
</blockquote>
<p>The pthread rule is, pthread_create() doen't introduce any process tree change. In the other words, kFreeBSD is buggy.</p>
<p>But I'm planning to commit your patch because 1) it's only test case change and no impact to ruby code base 2) your patch<br>
doesn't introduce any ugliness.</p> Ruby master - Bug #5239: bootstraptest/runner.rb: assert_normal_exit logic broken on Debian/GNU kFreeBSDhttps://redmine.ruby-lang.org/issues/5239?journal_id=204512011-08-28T06:29:09Zlucas (Lucas Nussbaum)lucas@lucas-nussbaum.net
<ul></ul><p>On 28/08/11 at 02:30 +0900, Motohiro KOSAKI wrote:</p>
<blockquote>
<p>Issue <a class="issue tracker-1 status-6 priority-4 priority-default closed" title="Bug: bootstraptest/runner.rb: assert_normal_exit logic broken on Debian/GNU kFreeBSD (Rejected)" href="https://redmine.ruby-lang.org/issues/5239">#5239</a> has been updated by Motohiro KOSAKI.</p>
<p>Status changed from Open to Assigned<br>
Assignee set to Motohiro KOSAKI<br>
Target version set to 1.9.3</p>
<blockquote>
<p>The problem is that the call to $? in the thread cannot retrieve the exit value of the process started by popen. "$?" is<br>
transformed into a wait4() syscall, but wait4 is not allowed to inquire the state of non-child processes. And the popen<br>
process is not a child of the sub-thread, it is a child of the main thread.</p>
<p><a href="http://pubs.opengroup.org/onlinepubs/009695399/functions/wait.html" class="external">http://pubs.opengroup.org/onlinepubs/009695399/functions/wait.html</a> confirms that waitpid() is not supposed to work on<br>
non-child processes.</p>
</blockquote>
<p>In fact, it's a child process. see more below paragraph.</p>
<blockquote>
<p>RATIONALE<br>
A call to the wait() or waitpid() function only returns status on an immediate child process of the <em>calling process</em></p>
</blockquote>
<p>The pthread rule is, pthread_create() doen't introduce any process tree change. In the other words, kFreeBSD is buggy.</p>
</blockquote>
<p>One could argue that a different PID makes a different process, but it's<br>
correct that it's not a different process.</p>
<blockquote>
<p>But I'm planning to commit your patch because 1) it's only test case change and no impact to ruby code base 2) your patch<br>
doesn't introduce any ugliness.</p>
</blockquote>
<p>Thanks</p>
<p>Lucas</p> Ruby master - Bug #5239: bootstraptest/runner.rb: assert_normal_exit logic broken on Debian/GNU kFreeBSDhttps://redmine.ruby-lang.org/issues/5239?journal_id=204542011-08-28T09:53:06Zkosaki (Motohiro KOSAKI)kosaki.motohiro@gmail.com
<ul><li><strong>ruby -v</strong> changed from <i>1.9.3</i> to <i>-</i></li></ul><blockquote>
<blockquote>
<blockquote>
<p>RATIONALE<br>
A call to the wait() or waitpid() function only returns status on an immediate child process of the <em>calling process</em></p>
</blockquote>
<p>The pthread rule is, pthread_create() doen't introduce any process tree change. In the other words, kFreeBSD is buggy.</p>
</blockquote>
<p>One could argue that a different PID makes a different process, but it's<br>
correct that it's not a different process.</p>
</blockquote>
<p>popen makes different process. and Thread.new don't make a process. isn't it?<br>
Therefore a subthread clealy can use waitpid() for waiting child<br>
process. Moreover<br>
real FreeBSD works this test completely. It's only kFreeBSD issue.</p>
<p>I wonder why kFreeBSD don't handle threading correctly. Long time ago,<br>
linuxthreads made the same mistake and application users sufferd from it.<br>
I didn't expected we observed the same issue at 21th century.</p> Ruby master - Bug #5239: bootstraptest/runner.rb: assert_normal_exit logic broken on Debian/GNU kFreeBSDhttps://redmine.ruby-lang.org/issues/5239?journal_id=204612011-08-29T01:18:38Zlucas (Lucas Nussbaum)lucas@lucas-nussbaum.net
<ul></ul><p>Hi,</p>
<p>I've just checked, and FreeBSD 8.2 is also affected by this issue.<br>
Test script:<br>
system("false")<br>
sleep 0.5<br>
th = Thread::new { $? }<br>
p th<br>
th.join<br>
p th<br>
p th.value</p> Ruby master - Bug #5239: bootstraptest/runner.rb: assert_normal_exit logic broken on Debian/GNU kFreeBSDhttps://redmine.ruby-lang.org/issues/5239?journal_id=204692011-08-29T13:59:06Zlucas (Lucas Nussbaum)lucas@lucas-nussbaum.net
<ul></ul><p>On 29/08/11 at 12:43 +0900, KOSAKI Motohiro wrote:</p>
<blockquote>
<blockquote>
<p>I've just checked, and FreeBSD 8.2 is also affected by this issue.<br>
Test script:<br>
system("false")<br>
sleep 0.5<br>
th = Thread::new { $? }<br>
p th<br>
th.join<br>
p th<br>
p th.value</p>
</blockquote>
<p>This is buggy. $? is thread local storage variable. last exit value<br>
was set to main thread.<br>
because system() was called from it. Therefore it fail even if run on linux.</p>
</blockquote>
<p>Right, sorry.</p>
<blockquote>
<p>In the other hand, on assert_normal_exit(), io.close is called from<br>
sub thread. it's a source<br>
of difference.</p>
</blockquote>
<p>Right, it changes the order of termination.</p>
<blockquote>
<p>Please try following code. it work on linux.</p>
<p>io = IO.popen("false")<br>
th = Thread.new {<br>
io.read<br>
io.close<br>
v = $?<br>
p v<br>
v<br>
}<br>
sleep 0.5</p>
<p>p th<br>
th.join<br>
p th<br>
p th.value</p>
</blockquote>
<p>Indeed, that works on FreeBSD 8.2:<br>
#<Process::Status: pid 1044 exit 1><br>
#<Thread:0x00000801182168 dead><br>
#<Thread:0x00000801182168 dead><br>
#<Process::Status: pid 1044 exit 1></p>
<ul>
<li>Lucas</li>
</ul> Ruby master - Bug #5239: bootstraptest/runner.rb: assert_normal_exit logic broken on Debian/GNU kFreeBSDhttps://redmine.ruby-lang.org/issues/5239?journal_id=204722011-08-29T15:23:07Zlucas (Lucas Nussbaum)lucas@lucas-nussbaum.net
<ul></ul><p>On 29/08/11 at 13:56 +0900, Lucas Nussbaum wrote:</p>
<blockquote>
<p>On 29/08/11 at 12:43 +0900, KOSAKI Motohiro wrote:</p>
<blockquote>
<blockquote>
<p>I've just checked, and FreeBSD 8.2 is also affected by this issue.<br>
Test script:<br>
system("false")<br>
sleep 0.5<br>
th = Thread::new { $? }<br>
p th<br>
th.join<br>
p th<br>
p th.value</p>
</blockquote>
<p>This is buggy. $? is thread local storage variable. last exit value<br>
was set to main thread.<br>
because system() was called from it. Therefore it fail even if run on linux.</p>
</blockquote>
<p>Right, sorry.</p>
<blockquote>
<p>In the other hand, on assert_normal_exit(), io.close is called from<br>
sub thread. it's a source<br>
of difference.</p>
</blockquote>
<p>Right, it changes the order of termination.</p>
</blockquote>
<p>Err, no, it doesn't. the popen process still ends before the thread. I<br>
don't see why io.close would be a source of difference?</p>
<p>In any case, it's a bug in Debian GNU/kfreebsd. With the following test<br>
case:<br>
<-------------<br>
#include <pthread.h><br>
#include <stdio.h><br>
#include <stdlib.h><br>
#include <unistd.h><br>
#include <sys/types.h><br>
#include <sys/wait.h></p>
<p>int retval;<br>
int pid;</p>
<p>void * thread_get_retval(void *arg) {<br>
if (waitpid(pid, &retval, 0) == -1) {<br>
perror("waitpid");<br>
exit(1);<br>
}<br>
printf("hello\n");<br>
return(0);<br>
}</p>
<p>int main() {<br>
pthread_t tid;<br>
if ((pid = fork()) == 0) {<br>
return 42;<br>
}<br>
printf("PID: %d\n", pid);</p>
<pre><code>if (pthread_create(&tid, NULL, thread_get_retval, NULL)) {
perror("pthread_create");
exit(1);
}
if (pthread_join( tid, NULL )) {
perror("pthread_join");
exit(1);
}
printf("Retval: %d\n", WEXITSTATUS(retval));
return 0;
</code></pre>
<p>}<br>
--------------></p>
<p>Debian GNU/Linux:<br>
PID: 27610<br>
hello<br>
Retval: 42</p>
<p>FreeBSD:<br>
PID: 1172<br>
hello<br>
Retval: 42</p>
<p>Debian GNU/kFreeBSD:<br>
./debian-kfreebsd-amd64:~# ./forkthread<br>
PID: 719<br>
waitpid: No child processes</p>
<p>Do you still want to apply my patch? I could apply it only in the Debian<br>
package if you prefer.</p>
<ul>
<li>Lucas</li>
</ul> Ruby master - Bug #5239: bootstraptest/runner.rb: assert_normal_exit logic broken on Debian/GNU kFreeBSDhttps://redmine.ruby-lang.org/issues/5239?journal_id=204732011-08-29T15:43:01Zkosaki (Motohiro KOSAKI)kosaki.motohiro@gmail.com
<ul><li><strong>Status</strong> changed from <i>Assigned</i> to <i>Rejected</i></li></ul> Ruby master - Bug #5239: bootstraptest/runner.rb: assert_normal_exit logic broken on Debian/GNU kFreeBSDhttps://redmine.ruby-lang.org/issues/5239?journal_id=204742011-08-29T15:53:07Zkosaki (Motohiro KOSAKI)kosaki.motohiro@gmail.com
<ul></ul><blockquote>
<blockquote>
<blockquote>
<p>In the other hand, on assert_normal_exit(), io.close is called from<br>
sub thread. it's a source<br>
of difference.</p>
</blockquote>
<p>Right, it changes the order of termination.</p>
</blockquote>
<p>Err, no, it doesn't. the popen process still ends before the thread. I<br>
don't see why io.close would be a source of difference?</p>
</blockquote>
<p>In ruby internal, system(), io.close do 1) wait to finish child process <em>and</em><br>
2) set a last exit code of a child process to caller thread's thread<br>
local variable.</p>
<p>In the other word, $? is a more magical variable than you did expect. ;-)</p>
<p>btw, please imazine what's happen if $? is true global variable, it's<br>
completely useless.</p>
<blockquote>
<p>Debian GNU/Linux:<br>
PID: 27610<br>
hello<br>
Retval: 42</p>
<p>FreeBSD:<br>
PID: 1172<br>
hello<br>
Retval: 42</p>
<p>Debian GNU/kFreeBSD:<br>
./debian-kfreebsd-amd64:~# ./forkthread<br>
PID: 719<br>
waitpid: No child processes</p>
<p>Do you still want to apply my patch? I could apply it only in the Debian<br>
package if you prefer.</p>
</blockquote>
<p>It would be better. Thanks.</p>