Bug #8123: Transcoding exception when using replace along with universal_newline

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: 1.9.3 and 2.0.0
• Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN

[k776 (Kieran Pilkington)]

The following can be reproduced on 1.9.3-p385, 1.9.3-p392, and 2.0.0-p0.

body, charset = "hey", "hey".encoding.to_s
body = body.encode('UTF-8', charset,
:invalid => :replace, :undef => :replace,
:replace => '', :universal_newline => true
)
Encoding::ConverterNotFoundError: code converter not found (universal_newline)

However, if you remove ":replace => '', ", then it works as intended.

Seems you can't use the two options together.

[jsc (Justin Collins)]

This code appears to be working (not raising an exception) as early as Ruby 2.1.10:

$ ruby -v
ruby 2.1.10p492 (2016-04-01 revision 54464) [x86_64-darwin17.0]
$ irb
2.1.10 :001 > body, charset = "hey", "hey".encoding.to_s
 => ["hey", "UTF-8"]
2.1.10 :002 > body = body.encode('UTF-8', charset,
2.1.10 :003 >     :invalid => :replace, :undef => :replace,
2.1.10 :004 >     :replace => '', :universal_newline => true
2.1.10 :005?>   )
 => "hey"

[duerst (Martin Dürst)]

• Related to Bug #12431: Strange behavior of String#encode('UTF-8', 'UTF-8', ...) when the encoding of the source string is not UTF-8 added

[duerst (Martin Dürst)]

jsc (Justin Collins) wrote:

This code appears to be working (not raising an exception) as early as Ruby 2.1.10:

This may be related to issue #12431 (Strange behavior of String#encode('UTF-8', 'UTF-8', ...) when the encoding of the source string is not UTF-8). This issue was fixed about 2.5 years ago (i.e. Ruby 2.4 timeframe), and the fix was backported back two versions (i.e. Ruby 2.3 and 2.2, if I count correctly).

I'm assigning this (even if maybe only temporarily) to Nobu, because he fixed issue #12431.

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed

It seems fixed by #11324.