Bug #9129: Regression in support for IPv6 literals in URIs with Net::HTTP

• Status: Closed
• Assignee: naruse (Yui NARUSE)
• Target version: 2.2.0
• ruby -v: ruby 2.0.0p247 (2013-06-27 revision 41674) [x86_64-darwin12.4.0]
• Backport: 2.0.0: DONE, 2.1: DONE

[kallistec (Daniel DeLeo)]

The proxy code in Net::HTTP constructs a URI string by concatenating the address with the port. If the address is IPv6, this is incorrect, the address should be surrounded by square brackets. My code is failing in the Net::HTTP#proxy_uri method. The error can be reproduced by:

ruby -r'net/http' -e 'Net::HTTP.new("::1", 80).proxy_uri'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/generic.rb:214:in initialize': the scheme http does not accept registry part: ::1:80 (or bad hostname?) (URI::InvalidURIError) from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/http.rb:84:in initialize'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:214:in new' from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:214:in parse'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:747:in parse' from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:994:in URI'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1027:in proxy_uri' from -e:1:in '

Though I've not tried to reproduce on current trunk, the implementation of Net::HTTP#proxy_uri is unchanged from 2.0.0p0 and current trunk.

My application's tests pass if I monkey patch that method as follows:

  def proxy_uri # :nodoc:
    ipv6_safe_addr = address.to_s.include?(":") ? "[#{address}]" : address
    @proxy_uri ||= URI("http://#{ipv6_safe_addr}:#{port}").find_proxy
  end

[drbrain (Eric Hodel)]

• Category set to lib
• Status changed from Open to Assigned
• Assignee set to naruse (Yui NARUSE)
• Backport changed from 1.9.3: UNKNOWN, 2.0.0: UNKNOWN to 1.9.3: DONTNEED, 2.0.0: UNKNOWN

=begin
I can't find a regression here as no prior version has #proxy_uri:

$ ruby19 -v -rnet/http -e 'Net::HTTP.new("::1", 80).proxy_uri'
ruby 1.9.3p374 (2013-01-15 revision 38858) [x86_64-darwin12.2.1]
-e:1:in <main>': undefined method proxy_uri' for #<Net::HTTP ::1:80 open=false> (NoMethodError)

IIRC net/http has always required square brackets for IPv6 addresses:

$ ruby -v -rnet/http -e 'Net::HTTP.new("[::1]", 80).proxy_uri'
ruby 2.0.0p247 (2013-06-27 revision 41674) [x86_64-darwin12.4.0]
$

This is consistent with URI:

$ ruby -ruri -e 'u = URI "http://[::1]"; u.host = "[::2]"; puts u'
http://[::2]
$ ruby -ruri -e 'u = URI "http://[::1]"; u.host = "::2"; puts u'
[…]/uri/generic.rb:605:in `check_host': bad component(expected host component): ::2 (URI::InvalidComponentError)

If you are using URI to figure out the host portion, use #host, not #hostname:

$ ruby -ruri -e 'u = URI "http://[::1]"; p u.host, u.hostname'
"[::1]"
"::1"

See also the URI::HTTP#host and #hostname documentation.

I have assigned this to naruse as he is Net::HTTP maintainer, maybe I am wrong.

=end

[kallistec (Daniel DeLeo)]

Hi Eric,
Earlier ruby versions handled proxy support differently (I'm not very familiar with either the old or new code), so my repro won't show that. The issue is that HTTP#proxy_uri is in the path for all HTTP requests, at least the way I'm using Net::HTTP. Here is a section of stack trace from my application:

/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/generic.rb:214:in initialize' /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/http.rb:84:in initialize'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:214:in new' /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:214:in parse'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:747:in parse' /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/uri/common.rb:994:in URI'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1027:in proxy_uri' /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1014:in proxy?'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:868:in connect' /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:862:in do_start'
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:851:in start' /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1367:in request'

If I understand correctly, the first argument to Net::HTTP.new should be an address that you can safely pass on to the socket layer code. For example, run this on Ruby 1.9.3:

ruby -rnet/http -e "Net::HTTP.new('::1', 80).get('/')" # works if you're running an HTTP server, Errno::ECONNREFUSED otherwise

Whereas this:

ruby -rnet/http -e "Net::HTTP.new('[::1]', 80).get('/')"

Gives you "getaddrinfo: nodename nor servname provided, or not known (SocketError)" because it's treating the whole thing with the brackets as the hostname/address.

It's true that in URIs you need to wrap an IPv6 literal in square brackets, which is why I believe this particular line of code is incorrect: https://github.com/ruby/ruby/blob/trunk/lib/net/http.rb#L1028

Thanks

[kallistec (Daniel DeLeo)]

I will add also that this example gives a SocketError (as I would expect it to and is consistent with 1.9.3) on Ruby 2.0:

ruby -rnet/http -e "Net::HTTP.new('[::1]', 80).get('/')"
/Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in initialize': getaddrinfo: nodename nor servname provided, or not known (SocketError) from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in open'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:878:in block in connect' from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/timeout.rb:52:in timeout'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:877:in connect' from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:862:in do_start'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:851:in start' from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1367:in request'
from /Users/ddeleo/.rbenv/versions/2.0.0-p247/lib/ruby/2.0.0/net/http.rb:1126:in get' from -e:1:in '

[naruse (Yui NARUSE)]

• Target version set to 2.1.0

[hsbt (Hiroshi SHIBATA)]

• Target version changed from 2.1.0 to 2.2.0

[naruse (Yui NARUSE)]

• Status changed from Assigned to Closed
• % Done changed from 0 to 100

Applied in changeset r47079.

---

• lib/net/http.rb (Net::HTTP.proxy_uri): use initializer instead
  of parser to handle IPv6 address. [Bug #9129]

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 1.9.3: DONTNEED, 2.0.0: UNKNOWN to 1.9.3: DONTNEED, 2.0.0: REQUIRED

according to #10531 it seems required to be backported into ruby_2_1.

[usa (Usaku NAKAMURA)]

• Related to Backport #10530: Backport r47079 to 2.0.0 added

[usa (Usaku NAKAMURA)]

• Backport changed from 1.9.3: DONTNEED, 2.0.0: REQUIRED to 2.0.0: DONE, 2.1: REQUIRED

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.0.0: DONE, 2.1: REQUIRED to 2.0.0: DONE, 2.1: DONE

Backported at r49076.