Backport #670: select doesn't handle fd's > FD_SETSIZE very well

• Status: Rejected
• Assignee: usa (Usaku NAKAMURA)

[rogerdpack (Roger Pack)]

=begin
The main motivation for reporting this bug is that currently on win32 [any version] if you try to pass more than 64 sockets to select, it ignores the 65th onward. This limit is not obeyed by Ruby currently and therefore some sockets will never select.

I'd first suggest raising an error if more than 64 total sockets are passed [to the most recent thread that called select?]

While examining it, this same aspect [too many or too high of numbered selects] is also seen to cause some problems on other platforms.

In the test cases attached, sometimes Mac OS X consistently crashes, sometimes Linux does. Depends on the test.

I'd bet that these problems also continue for Ruby 1.9 as well.

How to run tests:
first touch file 'abc' # used for some tests
then run ulimit -n 2000 # if this fails, you may need to run "sudo bash" first
then run the files.

To ruby's credit, without having first run the ulimit, above, most platforms don't seg fault [though windows still has the difficulty mentioned], so this isn't life critical.

Note that in Linux/OS X if a file descriptor's "number" is > FD_SETSIZE then it will be ignored by select. In Windows if the "total number of descriptors" past to select is > FD_SETSIZE then those past FD_SETSIZE will be ignored.

Thanks much!
-=Roger
=end

[rogerdpack (Roger Pack)]

=begin
http://redmine.ruby-lang.org/issues/show/755 patch helps with the 'hangs_windows' test case.
=end

[shyouhei (Shyouhei Urabe)]

• Assignee set to usa (Usaku NAKAMURA)

=begin

=end

[usa (Usaku NAKAMURA)]

• Priority changed from Normal to 3
• ruby -v set to -

=begin
When the patch is applied, every select call uses the stack by 112KB (about 1.8KB at present).
And, if you pass 4097 sockets to select, the same problem occurs again.
So this approach is not preferable.

In 1.9, this problem has already been solved, because we didn't have to consider the binary level compatibility.
But in 1.8, we should give priority to maintenance of compatibility more than such a rare case.

Of course, it is necessary to correct this problem if we can.
I leave this problem as Open until another effective ideas arise.
=end

[rogerdpack (Roger Pack)]

=begin
Two things come to mind, if 4096 is not an acceptable option :)

1. check for fd number count before doing a select. If it's > FD_SETSIZE raise error.
2. A compromise, ex: setting FD_SETSIZE to 512, since that's the limit of MSVCRT 6, which is used by 99% of 1.8.x compilers. I put 4096 as a conservative number.

Thanks for looking into this.

-=r
=end

[usa (Usaku NAKAMURA)]

• Status changed from Open to Assigned

=begin

=end

[rogerdpack (Roger Pack)]

=begin
Would it be possible to at least set FD_SETSIZE to 256 so that it works with VC6/mingw (which is 99% of ruby on windows distros)?

That would be good--currently there it is hard for extensions to use select correctly because they overflow the FD set and cannot reset it because it is hard compiled to too small on windows.
Thanks!
-r
=end

[rogerdpack (Roger Pack)]

• File socket.diff socket.diff added

=begin
Attaching new patch that has a smaller FD_SETSIZE, with test (fails without FD_SETSIZE, succeeds with).
Possible to commit this one, perhaps?
-r
=end

[rogerdpack (Roger Pack)]

=begin
Another option might be something like the following (any feedback?)
Thanks!

Index: win32/win32.h

--- win32/win32.h (revision 26621)
+++ win32/win32.h (working copy)
@@ -23,6 +23,9 @@
#ifdef BORLANDC
#define USE_WINSOCK2
#endif
+#ifndef FD_SETSIZE
+#if defined MINGW32
+# define FD_SETSIZE 256 // larger default for msvcrt 6
+#endif
+#endif
#ifdef USE_WINSOCK2
#include <winsock2.h>
#include <ws2tcpip.h>

=end

[usa (Usaku NAKAMURA)]

• Tracker changed from Bug to Backport
• Project changed from Ruby 1.8 to Backport186
• Description updated (diff)
• Status changed from Assigned to Rejected
• ruby -v deleted (-)

1.8.6 is out of date