Feature #14339
closed[PATCH] net/http: use writev for HTTP chunked request bodies
Description
net/http: use writev for HTTP chunked request bodies
This reduces both user and system CPU time for large
uploads with dynamically-generated request bodies.
user system total real
before: 0.393334 1.580000 1.973334 ( 1.971066)
after: 0.223334 0.976666 1.200000 ( 1.198514)
------
require 'socket'
require 'net/http'
require 'benchmark'
nr = 1024 * 1024 * 1024
s = TCPServer.new('127.0.0.1', 0)
addr = s.addr
at_exit { Process.waitall }
fork do
c = s.accept
# not exactly accurate but fast
IO.copy_stream(c, '/dev/null', nr + 500000)
begin
buf = c.readpartial(16384)
tmp = ''
until buf.end_with?(-"0\r\n\r\n")
buf << c.readpartial(16384, tmp)
end
rescue EOFError
end
c.write "HTTP/1.1 201 Created\r\nConnection:close\r\n\r\n"
c.close
end
r, w = IO.pipe
fork do
r.close
IO.copy_stream('/dev/zero', w, nr)
w.close
end
w.close
Net::HTTP.start(addr[3], addr[1]) do |http|
put = Net::HTTP::Put.new('/dev0/foo')
put['Content-Type'] = 'application/content-type'
put['Transfer-Encoding'] = 'chunked'
put.body_stream = r
puts(Benchmark.measure { http.request(put) })
end
------
* lib/net/http/generic_request.rb (write): use multi-arg write
* lib/net/protocol.rb (write): support multi-arg
(write0): ditto
Files
Updated by naruse (Yui NARUSE) over 6 years ago
@debug_output << Array(strs).map(&:dump).join if @debug_output
I thought strs is always Array.
Is Array() is needed here?
other parts are looks good to me.
Updated by normalperson (Eric Wong) over 6 years ago
naruse@airemix.jp wrote:
@debug_output << Array(strs).map(&:dump).join if @debug_output
I thought strs is always Array.
Is Array() is needed here?
No, will remove before committing.
other parts are looks good to me.
Btw, I noticed [Bug #14338] while working on this, any comments
on what .write() (no args) should be?? Thanks.
Updated by Anonymous over 6 years ago
- Status changed from Open to Closed
Applied in changeset trunk|r61812.
net/http: use writev for HTTP chunked request bodies
This reduces both user and system CPU time for large
uploads with dynamically-generated request bodies.
user system total real
before: 0.393334 1.580000 1.973334 ( 1.971066)
after: 0.223334 0.976666 1.200000 ( 1.198514)
require 'socket'
require 'net/http'
require 'benchmark'
nr = 1024 * 1024 * 1024
s = TCPServer.new('127.0.0.1', 0)
addr = s.addr
at_exit { Process.waitall }
fork do
c = s.accept
not exactly accurate but fast¶
IO.copy_stream(c, '/dev/null', nr + 500000)
begin
buf = c.readpartial(16384)
tmp = ''
until buf.end_with?(-"0\r\n\r\n")
buf << c.readpartial(16384, tmp)
end
rescue EOFError
end
c.write "HTTP/1.1 201 Created\r\nConnection:close\r\n\r\n"
c.close
end
r, w = IO.pipe
fork do
r.close
IO.copy_stream('/dev/zero', w, nr)
w.close
end
w.close
Net::HTTP.start(addr[3], addr[1]) do |http|
put = Net::HTTP::Put.new('/dev0/foo')
put['Content-Type'] = 'application/content-type'
put['Transfer-Encoding'] = 'chunked'
put.body_stream = r
puts(Benchmark.measure { http.request(put) })
end
- lib/net/http/generic_request.rb (write): use multi-arg write
- lib/net/protocol.rb (write): support multi-arg
(write0): ditto
[ruby-core:84845] [Feature #14339]