Bug #9986: WEBrick content-length being set when transfer-encoding is chunked

• Status: Closed
• Assignee: nahi (Hiroshi Nakamura)
• Target version: -
• ruby -v: trunk, ruby 2.1.2p95
• Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN

[lengarvey (Leonard Garvey)]

It's possible to get WEBrick to return both Transfer-Encoding: chunked and a calculated Content-length header. If the Transfer-encoding header is set via WEBrick::HTTPResponse#[]= then #chunked? will return false and the content length will be set during the setup_headers method. This causes issues with rack and safari (example code for rack can be seen https://github.com/rack/rack/issues/618#issuecomment-47355492). As far as I'm aware WEBrick shouldn't return a content-length when a transfer-encoding chunked header is present. Messages MUST NOT include both a Content-Length header field and a transfer-coding. as per http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html

The following test can be placed into test_httpresponse.rb to demonstrate the issue:

    def test_200_chunked_does_not_set_content_length
      res.body        = 'hello'
      res.status      = 200
      res.chunked     = false
      res["Transfer-Encoding"] = 'chunked'
      res.setup_header

      assert_nil res.header.fetch('content-length', nil)
    end

I've added a patchfile which includes the above test and a fix for the issue.

[nobu (Nobuyoshi Nakada)]

• Is duplicate of Bug #9927: webrick does not unset content-length when responding to HEAD requests added

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Rejected

[lengarvey (Leonard Garvey)]

I don't believe this is a duplicate of #9927. This occurs if you're not issuing a HEAD request and affects Safari with standard rack applications which use the Rack::Chunked middleware.

ᐅ curl -i http://localhost:8000
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Server: WEBrick/1.3.1 (Ruby/2.1.2/2014-05-08)
Date: Sat, 28 Jun 2014 01:50:46 GMT
Content-Length: 26
Connection: Keep-Alive

Hello world%

Which can be replicated with the following webrick server:

require 'webrick'

server = WEBrick::HTTPServer.new :Port => 8000

server.mount_proc '/' do |req, res|
  res.status      = 200
  res.chunked     = false
  res["Transfer-Encoding"] = 'chunked'
  res.body = "5\r\nHello\r\n6\r\n world\r\n0\r\n\r\n"
end

trap 'INT' do server.shutdown end
server.start

Note that this response includes the both the Transfer-Encoding: chunked header and the Content-Length header. This (to my understanding) is against RFC2616 Section 4(http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html) although I'd agree that Safari is also not complying with the spec too:

Messages MUST NOT include both a Content-Length header field and a non-identity transfer-coding. If the message does include a non- identity transfer-coding, the Content-Length MUST be ignored.

[naruse (Yui NARUSE)]

• Is duplicate of deleted (Bug #9927: webrick does not unset content-length when responding to HEAD requests)

[naruse (Yui NARUSE)]

• Status changed from Rejected to Assigned
• Target version set to 2.2.0

It sounds reasonable.
But the fix should be in WEBrick::HTTPResponse#[]=.

[lengarvey (Leonard Garvey)]

Thanks for taking another look.

It's a relatively simple fix either way, I made the change in HTTPResponse#chunked? because I assumed that less 3rd parties would use this method so would be less affected by the additional logic within. Although I guess this leaves the @chunked instance variable incorrect until queried using #chunked? which may cause issues down the line.

[lengarvey (Leonard Garvey)]

• File webrick_transfer_encoding_chunked_content_length_failing_test.patch webrick_transfer_encoding_chunked_content_length_failing_test.patch added
• File webrick_transfer_encoding_chunked_content_length_fix.patch webrick_transfer_encoding_chunked_content_length_fix.patch added

I've split out my patch into two separate patches. The first contains a failing test and the second contains a fix which has been applied to WEBrick::HTTPResponse#[]= instead of #chunked?

I assume this is the correct way of supplying these changes. I can easily open a pull request on github.com/ruby/ruby if that would be easier.

[naruse (Yui NARUSE)]

• Target version deleted (2.2.0)

[jeremyevans0 (Jeremy Evans)]

I added a pull request for this: https://github.com/ruby/webrick/pull/24

[jeremyevans0 (Jeremy Evans)]

• Status changed from Assigned to Closed