Bug #2554: Net::FTP should not use MSG_OOB

• Status: Rejected
• Assignee: shugo (Shugo Maeda)
• Target version: -
• ruby -v: ruby 1.8.7 (2009-12-24 patchlevel 248) [x86_64-linux]

[hongli (Hongli Lai)]

=begin
Net::FTP#abort, #status etc currently send commands with MSG_OOB. This causes a freeze when running RubySpec on x86_64 Linux (Ubuntu 9.10):

hongli@ubuntu:~/rubyspec$ ./mspec/bin/mspec library/net/ftp/abort_spec.rb -e "Net::FTP#abort sends the ABOR command to the server"
(watch it freeze)

This particular test sets up a dummy FTP server, connects to it with Net::FTP, and then calls Net::FTP#abort. Net::FTP#abort freezes while waiting for the FTP server response.

The dummy FTP server has a very simple and naive implementation. Its command handling main loop looks like this:

loop do
command = @socket.recv(1024)
if command is a QUIT command
break
else
handle_command(command)
end
end

The problem is that MSG_OOB only sends the last byte of the data as urgent data; the rest is sent as regular data. From http://popcnt.org/2007/07/what-happened-to-tcp-flag-urgent-msgoob.html:

Usually when socket receives tcp packet with URG flag it treats it as normal tcp data. recv() is going to read urgent data as it was normal tcp stream. The only difference is that the last byte of data is discarded. The last byte in urgent data was always a problem due to incoherent rfc.

Net::FTP#abort calls @sock.send("ABOR\r\n", Socket::MSG_OOB). This causes the server to receive "ABOR\r" (notice the lack of "\n"); the "\r" is discard from the normal TCP stream, and can only be read if the server reads the socket with MSG_OOB. However, after having read "ABOR\r", select() on x86_64 Linux still indicates that the socket is readable. This causes the server to call recv() without MSG_OOB on the socket again, and this recv() freezes because it cannot read the urgent data.

The FTP protocol description does not seem to make any mention that the ABOR command should be sent with MSG_OOB. This makes be believe that Net::FTP is bugged, not the server.

This freeze seems to be operating system specific and configure flag specific:

• The freeze occurs on Ruby 1.8.7-p248 on x86_64 Linux, when compiled from source with no extra configure flags.
• The freeze does not occur on the default Ruby as packaged by x86_64 Ubuntu 9.10. This is Ruby 1.8.7-p174, and I believe configured with --enable-pthreads --enable-shared.
• The freeze does not occur on OS X Snow Leopard.
  =end

[shugo (Shugo Maeda)]

• Status changed from Open to Assigned
• Assignee set to shugo (Shugo Maeda)

=begin
The use of MSG_OOB in Net::FTP is taken from ftplib.py in Python. If there is no objection, I will fix Net::FTP not to use MSG_OOB.
=end

[shugo (Shugo Maeda)]

=begin
Hi,

2010/1/6 Tanaka Akira [e-mail]:

The FTP protocol description does not seem to make any mention that the ABOR command should be sent with MSG_OOB. This makes be believe that Net::FTP is bugged, not the server.

Although FTP (RFC 959) doesn't mention OOB, the control connection follows
Telnet Protocol and TELNET (RFC 854) uses TCP Urgent notification.

I don't know the details of FTP and TELNET, though.

In page 34 to 35 of RFC 959, it is described as follows:

FTP commands may be partitioned as those specifying access-control
identifiers, data transfer parameters, or FTP service requests.
Certain commands (such as ABOR, STAT, QUIT) may be sent over the
control connection while a data transfer is in progress.  Some
servers may not be able to monitor the control and data connections
simultaneously, in which case some special action will be necessary
to get the server's attention.  The following ordered format is
tentatively recommended:

   1. User system inserts the Telnet "Interrupt Process" (IP) signal
   in the Telnet stream.

   2. User system sends the Telnet "Synch" signal.

   3. User system inserts the command (e.g., ABOR) in the Telnet
   stream.

   4. Server PI, after receiving "IP", scans the Telnet stream for
   EXACTLY ONE FTP command.

Then, ncftp seems to send the IP and Sync singals with MSG_OOB before
sending ABOR without MSB_OOB.
Should I fix Net::FTP in a similar way?

--
Shugo Maeda

=end

[shugo (Shugo Maeda)]

• Status changed from Assigned to Rejected

=begin
ftplib.py says:

 def abort(self):
     '''Abort a file transfer.  Uses out-of-band data.
     This does not follow the procedure from the RFC to send Telnet
     IP and Synch; that doesn't seem to work with the servers I've
     tried.  Instead, just send the ABOR command as OOB data.'''

So I suppose it's not good to change the behavior for dummy FTP servers.
=end