Backport #2553
closed
Fix pthreads slowness by eliminating unnecessary sigprocmask calls
Added by dpiddy (Dan Peterson) almost 15 years ago. Updated almost 8 years ago.
Description
=begin
This is a bug report for what's described here:
http://timetobleed.com/fix-a-bug-in-rubys-configurein-and-get-a-30-performance-boost/
Matz says here that this should already be fixed in 1.8.7:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/23583
But it does not appear to be. Example with 1.8.7-p248:
apply patch from above link¶
% make distclean
% ./configure --enable-pthread
% make
% time ./ruby -e '1_000_000.times { x = 2 ** 256 }'
./ruby -e '1_000_000.times { x = 2 ** 256 }' 1.45s user 0.34s system 99% cpu 1.809 total
% strace ./ruby -e '1_000.times { x = 2 ** 256 }' 2>&1 | grep -c sigproc
1009
% make distclean
% ./configure --disable-ucontext --enable-pthread
% make
% time ./ruby -e '1_000_000.times { x = 2 ** 256 }'
./ruby -e '1_000_000.times { x = 2 ** 256 }' 1.16s user 0.00s system 99% cpu 1.171 total
% strace ./ruby -e '1_000.times { x = 2 ** 256 }' 2>&1 | grep -c sigproc
3
% ./ruby -v
ruby 1.8.7 (2009-12-24 patchlevel 248) [i686-linux]
=end
Updated by dpiddy (Dan Peterson) almost 15 years ago
=begin
It's entirely possible this is a duplicate but after 10 minutes searching via both the Redmine interface and Google focused on this site I couldn't find it already reported.
=end
Updated by naruse (Yui NARUSE) almost 15 years ago
- Status changed from Open to Assigned
- Assignee set to matz (Yukihiro Matsumoto)
=begin
=end
Updated by dpiddy (Dan Peterson) over 14 years ago
=begin
Anything else I can do to help this along?
=end
Updated by rogerdpack (Roger Pack) over 14 years ago
=begin
I thought this was fixed in 1.8.7
Could you verify which branches (1.8.8, 1.8.7, 1.8.6) still show this problem?
That might help.
=end
Updated by kc7zzv (Patrick Mohr) over 14 years ago
=begin
I thought this was fixed in 1.8.7
Could you verify which branches (1.8.8, 1.8.7, 1.8.6) still show this problem?
I just ran the same tests on 1.8.6 and 1.8.7. It's still not fixed. I'm not sure where the source code for 1.8.8 is so I didn't test it.
The build information and test results are below. I didn't test 1.8.7 with --disable-ucontext because that configure flag doesn't seem to exist anymore.
Admittedly, ruby 1.8.7 is much faster than 1.8.6, but 1.8.7 still has this bug.
I would be happy to test 1.8.8 if you would tell me where to get the source code.
1.8.6 with pthreads test results:
svn co http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8_6
autoconf && ./configure --enable-pthread && make¶
./ruby -v¶
ruby 1.8.6 (2010-06-12 patchlevel 415) [i686-linux]
time ./ruby -e '1_000_000.times { x = 2 ** 256 }'¶
real 0m6.462s
user 0m6.128s
sys 0m0.308s
1.8.6 with pthreads enabled and ucontext disabled
svn co http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8_6
autoconf && ./configure --disable-ucontext --enable-pthread && make¶
./ruby -v¶
ruby 1.8.6 (2010-06-12 patchlevel 415) [i686-linux]
time ./ruby -e '1_000_000.times { x = 2 ** 256 }'¶
real 0m5.582s
user 0m5.548s
sys 0m0.000s
1.8.7 with pthreads
svn co http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8_7
autoconf && ./configure --enable-pthread && make¶
./ruby -v¶
ruby 1.8.7 (2010-06-16 patchlevel 296) [i686-linux]
time ./ruby -e '1_000_000.times { x = 2 ** 256 }'¶
real 0m3.241s
user 0m2.960s
sys 0m0.260s
1.8.7 with --disable-pthread
svn co http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8_7
autoconf && ./configure --disable-pthread && make¶
./ruby -v¶
ruby 1.8.7 (2010-06-16 patchlevel 296) [i686-linux]
time ./ruby -e '1_000_000.times { x = 2 ** 256 }'¶
real 0m2.535s
user 0m2.512s
sys 0m0.004s
=end
Updated by kosaki (Motohiro KOSAKI) over 14 years ago
=begin
ruby 1.8.8 have the same issue.
$ autoconf && ./configure --disable-pthread && make
$ time ./ruby -ve '1_000_000.times { x = 2 ** 256 }'
ruby 1.8.8dev (2010-06-15) [x86_64-linux]
real 0m1.176s
user 0m1.171s
sys 0m0.005s
$ autoconf && ./configure --enable-pthread && make
[kosaki@kosaopt ruby_1_8]$ time ./ruby -ve '1_000_000.times { x = 2 ** 256 }'
ruby 1.8.8dev (2010-06-15) [x86_64-linux]
real 0m1.513s
user 0m1.340s
sys 0m0.173s
$ strace -c ./ruby -e '1_000_000.times { x = 2 ** 256 }'
% time seconds usecs/call calls errors syscall
99.98 0.160160 0 1000011 rt_sigprocmask
0.02 0.000025 2 14 mprotect
0.00 0.000000 0 8 read
0.00 0.000000 0 19 10 open
0.00 0.000000 0 9 close
0.00 0.000000 0 4 3 stat
0.00 0.000000 0 9 fstat
0.00 0.000000 0 26 mmap
0.00 0.000000 0 1 munmap
0.00 0.000000 0 7 brk
0.00 0.000000 0 14 rt_sigaction
0.00 0.000000 0 1 1 access
0.00 0.000000 0 1 execve
0.00 0.000000 0 6 getrlimit
0.00 0.000000 0 1 getuid
0.00 0.000000 0 1 getgid
0.00 0.000000 0 2 geteuid
0.00 0.000000 0 2 getegid
0.00 0.000000 0 1 arch_prctl
0.00 0.000000 0 2 1 futex
0.00 0.000000 0 1 set_tid_address
0.00 0.000000 0 1 set_robust_list
100.00 0.160185 1000141 15 total
summary of ruby_1_8 code
node.h¶
#if defined(HAVE_GETCONTEXT) && defined(HAVE_SETCONTEXT)
#include <ucontext.h>
#define USE_CONTEXT
#endif
#include <setjmp.h>
#include "st.h"
The syscam can use getcontext(), USE_CONTEXT is always set.
eval.c¶
#ifdef USE_CONTEXT
(snip)
define ruby_setjmp(just_before_setjmp, j) ((j)->status = 0, \¶
(just_before_setjmp), \
PRE_GETCONTEXT, \
getcontext(&(j)->context), \
POST_GETCONTEXT, \
(j)->status)
if USE_CONTEXT is defined, ruby_setjmp() mean to call getcontext().
and linux getcontext() implementation call rt_sigprocmask() internally.
=end
Updated by kosaki (Motohiro KOSAKI) over 14 years ago
=begin
note: 1.9.x tree don't have this issue beucause ruby_setjmp() don't use getcontext().
=end
Updated by kosaki (Motohiro KOSAKI) over 14 years ago
=begin
This seems to be introduced following commit.
commit 9fdbc41973971f4ef032d57c4e02ed0430227a0a
Author: nobu nobu@b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Date: Sat Dec 13 00:01:28 2003 +0000
* configure.in: check ucontext.h.
* eval.c: use getcontext/setcontext() instead of setjmp/longjmp()
on ia64 or with native thread enabled. [ruby-core:01932]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@5180 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
but unfortunatelly, [ruby-core:01932] has alomost zero information ;)
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/1932
=end
Updated by rogerdpack (Roger Pack) over 14 years ago
=begin
Thanks for looking into this one.
-r
=end
Updated by kosaki (Motohiro KOSAKI) over 14 years ago
=begin
I and nobu digged previous discussion awhile. Now, we have concluded getcontext() is only necessary when following cases
- Using linuxthreads (it has broken signal model)
- cpu arch has register stack
=end
Updated by kosaki (Motohiro KOSAKI) over 14 years ago
- Category set to core
- Assignee changed from matz (Yukihiro Matsumoto) to shyouhei (Shyouhei Urabe)
- Priority changed from Normal to 3
=begin
This issue has been fixed by commit r28404 (ruby_1_8 branch).
=end
Updated by andrenth (Andre Nathan) over 14 years ago
=begin
Any chance of backporting the fix to the ruby_1_8_7 branch?
Thanks,
Andre
=end
Updated by kosaki (Motohiro KOSAKI) over 14 years ago
=begin
There is.
But unfortunately 1.8.7-p299 was released very recently. So you need to wait about half year ;-)
Also, If anyone will find a bug in this patch, our chance will disappear....
=end
Updated by shyouhei (Shyouhei Urabe) over 14 years ago
=begin
Oh, sorry for the inconvenience... I was not aware of this. It should be backported.
=end
Updated by andrenth (Andre Nathan) over 14 years ago
=begin
Hello
I was trying to update the ubuntu ruby 1.8.7 package with this patch, but I'm still seeing the bad performance. The patch has the following line:
if test -n "(/lib/libc.so.6 2>/dev/null | fgrep 'Native POSIX Threads') 2> /dev/null"; then
use_context=yes
fi
Isn't the use of setcontext necessary only for linuxthreads, i.e. if the system doesn't have native posix threads? In that case, shouldn't the test be "test -z", so that use_context is set to yes if "Native POSIX Threads" does not appear in that output?
Apologies if I have misunderstood the patch.
Best regards,
Andre
=end
Updated by kosaki (Motohiro KOSAKI) over 14 years ago
=begin
2010/7/10 Andre Nathan redmine@ruby-lang.org:
Issue #2553 has been updated by Andre Nathan.
Hello
I was trying to update the ubuntu ruby 1.8.7 package with this patch, but I'm still seeing the bad performance. The patch has the following line:
if test -n "
(/lib/libc.so.6 2>/dev/null | fgrep 'Native POSIX Threads') 2> /dev/null"; then
use_context=yes
fiIsn't the use of setcontext necessary only for linuxthreads, i.e. if the system doesn't have native posix threads? In that case, shouldn't the test be "test -z", so that use_context is set to yes if "Native POSIX Threads" does not appear in that output?
Apologies if I have misunderstood the patch.
Oops, this patch only works on 32bit machine, because some 64bit
distro don't have /lib/libc.so.6 (instead they use /lib64), and this
condition select use_context=yes.
I'll update this soon.
And, if you are using 32bit distro, please show your /lib/libc.so.6
execution result.
I'm very sorry this.
=end
Updated by kosaki (Motohiro KOSAKI) over 14 years ago
=begin
2010/7/10 KOSAKI Motohiro kosaki.motohiro@gmail.com:
2010/7/10 Andre Nathan redmine@ruby-lang.org:
Issue #2553 has been updated by Andre Nathan.
Hello
I was trying to update the ubuntu ruby 1.8.7 package with this patch, but I'm still seeing the bad performance. The patch has the following line:
if test -n "
(/lib/libc.so.6 2>/dev/null | fgrep 'Native POSIX Threads') 2> /dev/null"; then
use_context=yes
fiIsn't the use of setcontext necessary only for linuxthreads, i.e. if the system doesn't have native posix threads? In that case, shouldn't the test be "test -z", so that use_context is set to yes if "Native POSIX Threads" does not appear in that output?
Apologies if I have misunderstood the patch.
Oops, this patch only works on 32bit machine, because some 64bit
distro don't have /lib/libc.so.6 (instead they use /lib64), and this
condition select use_context=yes.I'll update this soon.
And, if you are using 32bit distro, please show your /lib/libc.so.6
execution result.I'm very sorry this.
following patch works both 32bit and 64bit on my environment.
$ svn diff
Index: configure.in
--- configure.in (リビジョン 28594)
+++ configure.in (作業コピー)
@@ -1151,7 +1151,7 @@
if test x"$rb_with_pthread" = xyes; then
AS_CASE("$target_cpu:$target_os:$cross_compiling",
[:linux:no], [
-
if test -n "`(/lib/libc.so.6 2>/dev/null | fgrep 'Native
POSIX Threads') 2> /dev/null`"; then
-
if test -n "`(/lib/libc.so.6 2>/dev/null | fgrep
'linuxthreads') 2> /dev/null`"; then
use_context=yes
fi
],
=end
Updated by kosaki (Motohiro KOSAKI) over 14 years ago
=begin
following patch works both 32bit and 64bit on my environment.
Committed.
Revision 28595 and 28597.
Thanks Andre, you are great.
=end
Updated by andrenth (Andre Nathan) over 14 years ago
=begin
The new patch fixed this issue for me. Thanks a lot!
Here's hoping that the patch will be merged into ruby_1_8_7. I believe it'll make it easier for distributions who package this version to have this performance bug fixed.
Thanks again!
Andre
=end
Updated by shyouhei (Shyouhei Urabe) about 14 years ago
- Status changed from Assigned to Closed
- % Done changed from 0 to 100
=begin
This issue was solved with changeset r29854.
Dan, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.
=end
Updated by shyouhei (Shyouhei Urabe) almost 14 years ago
- Tracker changed from Bug to Backport
- Status changed from Closed to Open
- % Done changed from 100 to 0
=begin
As of 1.8.7-334 this issue is reopened because the proposed fix broke binary compatibility. cf [ruby-dev:43152]
=end
Updated by shyouhei (Shyouhei Urabe) almost 8 years ago
- Status changed from Open to Closed