Feature #4142: multipart/form-data for net/http - Ruby master - Ruby Issue Tracking System

Actions

Copy link

Feature #4142

closed

multipart/form-data for net/http

Added by naruse (Yui NARUSE) over 13 years ago. Updated almost 13 years ago.

Status:

Closed

Assignee:

Target version:

[ruby-dev:42729]

Description

=begin
multipart/form-data 対応を net/http に入れませんか。
追加される API は Net::HTTPRequest#set_form になります。

akr さんからは multipart/form-data 用のデータを出力する API 案も示唆されたのですが、
chunked encoding を考慮に入れるとうまくまとまらなかったので見送っています。

diff --git a/lib/net/http.rb b/lib/net/http.rb
index 4d475b1..2751f77 100644
--- a/lib/net/http.rb
+++ b/lib/net/http.rb
@@ -22,6 +22,7 @@
require 'net/protocol'
autoload :OpenSSL, 'openssl'
require 'uri'
+autoload :SecureRandom, 'securerandom'

module Net #:nodoc:

@@ -1772,7 +1773,8 @@ module Net #:nodoc:
alias content_type= set_content_type

  # Set header fields and a body from HTML form data.

+params+ should be a Hash containing HTML form data.¶

+params+ should be an Array of Arrays or¶
a Hash containing HTML form data.¶

Optional argument +sep+ means data record separator.¶

Values are URL encoded as necessary and the content-type is set to¶

@@ -1792,6 +1794,48 @@ module Net #:nodoc:

  alias form_data= set_form_data

Set a HTML form data set.¶
+params+ is the form data set; it is an Array of Arrays or a Hash¶
+enctype is the type to encode the form data set.¶
It is application/x-www-form-urlencoded or multipart/form-data.¶
+formpot+ is an optional hash to specify the detail.¶
boundary:: the boundary of the multipart message¶
charset:: the charset of the message. All names and the values of¶
non-file fields are encoded as the charset.¶
Each item of params is an array and contains following items:¶
+name+:: the name of the field¶
+value+:: the value of the field, it should be a String or a File¶
+opt+:: an optional hash to specify additional information¶
Each item is a file field or a normal field.¶
If +value+ is a File object or the +opt+ have a filename key,¶
the item is treated as a file field.¶
If Transfer-Encoding is set as chunked, this send the request in¶
chunked encoding. Because chunked encoding is HTTP/1.1 feature,¶
you must confirm the server to support HTTP/1.1 before sending it.¶
Example:¶
http.set_form([["q", "ruby"], ["lang", "en"]])¶
See also RFC 2388, RFC 2616, HTML 4.01, and HTML5¶
def set_form(params, enctype='application/x-www-form-urlencoded', formopt={})
```
 @body_data = params
```
```
 @body = nil
```
```
 @body_stream = nil
```
```
 @form_option = formopt
```
```
 case enctype
```

 when /\Aapplication\/x-www-form-urlencoded\z/i,

```
   /\Amultipart\/form-data\z/i
```
```
   self.content_type = enctype
```
```
 else
```

   raise ArgumentError, "invalid enctype: #{enctype}"

```
 end
```
end
Set the Authorization: header for "Basic" authorization.¶

def basic_auth(account, password)
@header['authorization'] = [basic_encode(account, password)]
@@ -1849,6 +1893,7 @@ module Net #:nodoc:
self['User-Agent'] ||= 'Ruby'
@body = nil
@body_stream = nil
```
 @body_data = nil
```
end

attr_reader :method
@@ -1876,6 +1921,7 @@ module Net #:nodoc:
def body=(str)
@body = str
@body_stream = nil
```
 @body_data = nil
 str
```
end

@@ -1884,6 +1930,7 @@ module Net #:nodoc:
def body_stream=(input)
@body = nil
@body_stream = input

```
 @body_data = nil
 input
```
end

@@ -1901,6 +1948,8 @@ module Net #:nodoc:
send_request_with_body sock, ver, path, @body
elsif @body_stream
send_request_with_body_stream sock, ver, path, @body_stream

```
 elsif @body_data
```

   send_request_with_body_data sock, ver, path, @body_data
 else
   write_header sock, ver, path
 end

@@ -1935,6 +1984,92 @@ module Net #:nodoc:
end
end

def send_request_with_body_data(sock, ver, path, params)

 if /\Amultipart\/form-data\z/i !~ self.content_type

   self.content_type = 'application/x-www-form-urlencoded'

   return send_request_with_body(sock, ver, path, URI.encode_www_form(params))

```
 end
```
```
 opt = @form_option.dup
```

 opt[:boundary] ||= SecureRandom.urlsafe_base64(40)

 self.set_content_type(self.content_type, boundary: opt[:boundary])

```
 if chunked?
```
```
   write_header sock, ver, path
```

   encode_multipart_form_data(sock, params, opt)

```
 else
```
```
   require 'tempfile'
```
```
   file = Tempfile.new('multipart')
```

   encode_multipart_form_data(file, params, opt)

```
   file.rewind
```
```
   self.content_length = file.size
```
```
   write_header sock, ver, path
```
```
   IO.copy_stream(file, sock)
```
```
 end
```
end
def encode_multipart_form_data(out, params, opt)
```
 charset = opt[:charset]
```
```
 boundary = opt[:boundary]
```

 boundary ||= SecureRandom.urlsafe_base64(40)

```
 chunked_p = chunked?
```
```
 buf = ''
```
```
 params.each do |key, value, h={}|
```
```
   key = quote_string(key, charset)
```
```
   filename =
```

     h.key?(:filename) ? h[:filename] :

     value.respond_to?(:to_path) ? File.basename(value.to_path) :

```
     nil
```
```
   buf << "--#{boundary}\r\n"
```
```
   if filename
```

     filename = quote_string(filename, charset)

     type = h[:content_type] || 'application/octet-stream'

     buf << "Content-Disposition: form-data; " \

       "name=\"#{key}\"; filename=\"#{filename}\"\r\n" \

```
       "Content-Type: #{type}\r\n\r\n"
```

     if !out.respond_to?(:write) || !value.respond_to?(:read)

       # if +out+ is not an IO or +value+ is not an IO

       buf << (value.respond_to?(:read) ? value.read : value)

     elsif value.respond_to?(:size) && chunked_p

       # if +out+ is an IO and +value+ is a File, use IO.copy_stream

       flush_buffer(out, buf, chunked_p)

       out << "%x\r\n" % value.size if chunked_p

```
       IO.copy_stream(value, out)
```
```
       out << "\r\n" if chunked_p
```
```
     else
```

       # +out+ is an IO, and +value+ is not a File but an IO

       flush_buffer(out, buf, chunked_p)

       1 while flush_buffer(out, value.read(4096), chunked_p)

```
     end
```
```
   else
```
```
     # non-file field:
```

     #   HTML5 says, "The parts of the generated multipart/form-data

     #   resource that correspond to non-file fields must not have a

     #   Content-Type header specified."

     buf << "Content-Disposition: form-data; name=\"#{key}\"\r\n\r\n"

     buf << (value.respond_to?(:read) ? value.read : value)

```
   end
```
```
   buf << "\r\n"
```
```
 end
```
```
 buf << "--#{boundary}--\r\n"
```
```
 flush_buffer(out, buf, chunked_p)
```
```
 out << "0\r\n\r\n" if chunked_p
```
end
def quote_string(str, charset)

 str = str.encode(charset, fallback:->(c){'&#%d;'%c.encode("UTF-8").ord}) if charset

```
 str = str.gsub(/[\\"]/, '\\\\\&')
```
end
def flush_buffer(out, buf, chunked_p)
```
 return unless buf
```

 out << "%x\r\n"%buf.bytesize if chunked_p

```
 out << buf
```
```
 out << "\r\n" if chunked_p
```
```
 buf.clear
```
end
def supply_default_content_type
return if content_type()
warn 'net/http: warning: Content-Type did not set; using application/x-www-form-urlencoded' if $VERBOSE
diff --git a/lib/net/protocol.rb b/lib/net/protocol.rb
index 2a6cfb4..a3ffa71 100644
--- a/lib/net/protocol.rb
+++ b/lib/net/protocol.rb
@@ -168,6 +168,8 @@ module Net # :nodoc:
}
end
alias << write
def writeline(str)
writing {
write0 str + "\r\n"
diff --git a/test/net/http/test_http.rb b/test/net/http/test_http.rb
index 76280ad..12c03a4 100644
--- a/test/net/http/test_http.rb
+++ b/test/net/http/test_http.rb
@@ -303,6 +303,102 @@ module TestNetHTTP_version_1_2_methods
assert_equal data.size, res.body.size
assert_equal data, res.body
end
def test_set_form
require 'tempfile'
file = Tempfile.new('ruby-test')
file << "\u{30c7}\u{30fc}\u{30bf}"
data = [
```
 ['name', 'Gonbei Nanashi'],
```

 ['name', "\u{540d}\u{7121}\u{3057}\u{306e}\u{6a29}\u{5175}\u{885b}"],

 ['s"i\o', StringIO.new("\u{3042 3044 4e9c 925b}")],

```
 ["file", file, filename: "ruby-test"]
```
]
expected = <<"EOM".gsub(/\n/, "\r\n")
+--
+Content-Disposition: form-data; name="name"

+Gonbei Nanashi
+--
+Content-Disposition: form-data; name="name"
+
+\xE5\x90\x8D\xE7\x84\xA1\xE3\x81\x97\xE3\x81\xAE\xE6\xA8\xA9\xE5\x85\xB5\xE8\xA1\x9B
+--
+Content-Disposition: form-data; name="s\"i\\o"
+
+\xE3\x81\x82\xE3\x81\x84\xE4\xBA\x9C\xE9\x89\x9B
+--
+Content-Disposition: form-data; name="file"; filename="ruby-test"
+Content-Type: application/octet-stream
+
+\xE3\x83\x87\xE3\x83\xBC\xE3\x82\xBF
+----
+EOM

start {|http|

 _test_set_form_urlencoded(http, data.reject{|k,v|!v.is_a?(String)})

 _test_set_form_multipart(http, false, data, expected)

 _test_set_form_multipart(http, true, data, expected)

}
end
def _test_set_form_urlencoded(http, data)
req = Net::HTTP::Post.new('/')
req.set_form(data)
res = http.request req
assert_equal "name=Gonbei+Nanashi&name=%E5%90%8D%E7%84%A1%E3%81%97%E3%81%AE%E6%A8%A9%E5%85%B5%E8%A1%9B", res.body
end
def _test_set_form_multipart(http, chunked_p, data, expected)
data.each{|k,v|v.rewind rescue nil}
req = Net::HTTP::Post.new('/')
req.set_form(data, 'multipart/form-data')
req['Transfer-Encoding'] = 'chunked' if chunked_p
res = http.request req
body = res.body
assert_match(/\A--(?\S+)/, body)
/\A--(?\S+)/ =~ body
expected = expected.gsub(//, boundary)
assert_equal(expected, body)
end
def test_set_form_with_file
require 'tempfile'
file = Tempfile.new('ruby-test')
file << $test_net_http_data
filename = File.basename(file.to_path)
data = [['file', file]]
expected = <<"EOM".gsub(/\n/, "\r\n")
+--
+Content-Disposition: form-data; name="file"; filename=""
+Content-Type: application/octet-stream

+
+----
+EOM

expected.sub!(//, filename)
expected.sub!(//, $test_net_http_data)
start {|http|
```
 data.each{|k,v|v.rewind rescue nil}
```
```
 req = Net::HTTP::Post.new('/')
```

 req.set_form(data, 'multipart/form-data')

```
 res = http.request req
```
```
 body = res.body
```
```
 header, _ = body.split(/\r\n\r\n/, 2)
```

 assert_match(/\A--(?<boundary>\S+)/, body)

```
 /\A--(?<boundary>\S+)/ =~ body
```

 expected = expected.gsub(/<boundary>/, boundary)

 assert_match(/^--(?<boundary>\S+)\r\n/, header)

```
 assert_match(
```

   /^Content-Disposition: form-data; name="file"; filename="#{filename}"\r\n/,

```
   header)
```
```
 assert_equal(expected, body)
```
```
 data.each{|k,v|v.rewind rescue nil}
```
```
 req['Transfer-Encoding'] = 'chunked'
```
```
 res = http.request req
```
```
 #assert_equal(expected, res.body)
```
}
end
end

class TestNetHTTP_version_1_1 < Test::Unit::TestCase
=end

Actions

Copy link

Also available in: Atom PDF

Like0

Like0Like0Like0Like0

Project

General

Profile

Ruby » Ruby master

Custom queries

Feature #4142

multipart/form-data for net/http

+params+ should be a Hash containing HTML form data.¶

+params+ should be an Array of Arrays or¶

a Hash containing HTML form data.¶

Optional argument +sep+ means data record separator.¶

Values are URL encoded as necessary and the content-type is set to¶

Set a HTML form data set.¶

+params+ is the form data set; it is an Array of Arrays or a Hash¶

+enctype is the type to encode the form data set.¶

It is application/x-www-form-urlencoded or multipart/form-data.¶

+formpot+ is an optional hash to specify the detail.¶

boundary:: the boundary of the multipart message¶

charset:: the charset of the message. All names and the values of¶

non-file fields are encoded as the charset.¶

Each item of params is an array and contains following items:¶

+name+:: the name of the field¶

+value+:: the value of the field, it should be a String or a File¶

+opt+:: an optional hash to specify additional information¶

Each item is a file field or a normal field.¶

If +value+ is a File object or the +opt+ have a filename key,¶

the item is treated as a file field.¶

If Transfer-Encoding is set as chunked, this send the request in¶

chunked encoding. Because chunked encoding is HTTP/1.1 feature,¶

you must confirm the server to support HTTP/1.1 before sending it.¶

Example:¶

http.set_form([["q", "ruby"], ["lang", "en"]])¶

See also RFC 2388, RFC 2616, HTML 4.01, and HTML5¶

Set the Authorization: header for "Basic" authorization.¶

Updated by naruse (Yui NARUSE) over 13 years ago

Updated by nahi (Hiroshi Nakamura) almost 13 years ago

Updated by nahi (Hiroshi Nakamura) almost 13 years ago

Updated by naruse (Yui NARUSE) almost 13 years ago