Bug #9415: Strings#codepoints doesn't respect BOM on UTF-{16,32} pseudo encodings

• Status: Closed
• Assignee: naruse (Yui NARUSE)
• Target version: 2.2.0
• ruby -v: -
• Backport: 1.9.3: REQUIRED, 2.0.0: REQUIRED, 2.1: DONE

[nobu (Nobuyoshi Nakada)]

String#codepointsがUTF-16、UTF-32でのBOMを考慮していません。

$ ruby -e 'puts "%x" % "\u{feff}".encode("UTF-16BE").force_encoding("UTF-16").codepoints'
feff
$ ruby -e 'puts "%x" % "\u{feff}".encode("UTF-16LE").force_encoding("UTF-16").codepoints'
fffe

String#ordなども同様です。

$ ruby -e 'printf "%x\n", "\u{feff}".encode("UTF-16BE").force_encoding("UTF-16").ord'
feff
$ ruby -e 'printf "%x\n", "\u{feff}".encode("UTF-16LE").force_encoding("UTF-16").ord'
fffe

[nobu (Nobuyoshi Nakada)]

• ruby -v changed from r44601 to -

チケット #9415 が Nobuyoshi Nakada によって報告されました。

---

Bug #9415: Strings#codepoints doesn't respect BOM on UTF-{16,32} pseudo encodings
https://bugs.ruby-lang.org/issues/9415

• 作成者: Nobuyoshi Nakada
• ステータス: Open
• 優先度: Normal
• 担当者: Yui NARUSE
• カテゴリ: M17N
• 対象バージョン: current: 2.2.0
• ruby -v: r44601
• Backport: 1.9.3: UNKNOWN, 2.0.0: UNKNOWN, 2.1: REQUIRED

---

String#codepointsがUTF-16、UTF-32でのBOMを考慮していません。

$ ruby -e 'puts "%x" % "\u{feff}".encode("UTF-16BE").force_encoding("UTF-16").codepoints'
feff
$ ruby -e 'puts "%x" % "\u{feff}".encode("UTF-16LE").force_encoding("UTF-16").codepoints'
fffe

String#ordなども同様です。

$ ruby -e 'printf "%x\n", "\u{feff}".encode("UTF-16BE").force_encoding("UTF-16").ord'
feff
$ ruby -e 'printf "%x\n", "\u{feff}".encode("UTF-16LE").force_encoding("UTF-16").ord'
fffe

--
http://bugs.ruby-lang.org/

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed
• % Done changed from 0 to 100

Applied in changeset r44606.

---

string.c: respect BOM

• string.c (get_encoding): respect BOM on pseudo encodings.
  [ruby-dev:47895] [Bug #9415]

[nobu (Nobuyoshi Nakada)]

• Related to Bug #8940: printing UTF-32 crashs ruby added

[usa (Usaku NAKAMURA)]

• Backport changed from 1.9.3: UNKNOWN, 2.0.0: UNKNOWN, 2.1: REQUIRED to 1.9.3: REQUIRED, 2.0.0: REQUIRED, 2.1: REQUIRED

[naruse (Yui NARUSE)]

• Backport changed from 1.9.3: REQUIRED, 2.0.0: REQUIRED, 2.1: REQUIRED to 1.9.3: REQUIRED, 2.0.0: REQUIRED, 2.1: DONE

r45074