https://redmine.ruby-lang.org/https://redmine.ruby-lang.org/favicon.ico?17113305112014-09-27T15:07:21ZRuby Issue Tracking SystemRuby master - Bug #10300: Encoding error in conversion from UTF-16LE to UTF-8 to CP850https://redmine.ruby-lang.org/issues/10300?journal_id=491192014-09-27T15:07:21Zluislavena (Luis Lavena)luislavena@gmail.com
<ul><li><strong>Subject</strong> changed from <i>Troubles installing gems on Windows 7</i> to <i>Encoding error in conversion from UTF-16LE to UTF-8 to CP850</i></li></ul> Ruby master - Bug #10300: Encoding error in conversion from UTF-16LE to UTF-8 to CP850https://redmine.ruby-lang.org/issues/10300?journal_id=491202014-09-27T23:55:37Zduerst (Martin Dürst)duerst@it.aoyama.ac.jp
<ul></ul><p>There is no bug in the conversion from (UTF-16LE to) UTF-8 to CP850. CP850 simply doesn't contain U+2019 (RIGHT SINGLE QUOTATION MARK, see <a href="http://www.unicode.org/charts/PDF/U2000.pdf" class="external">http://www.unicode.org/charts/PDF/U2000.pdf</a>), see e.g. <a href="https://en.wikipedia.org/wiki/Code_page_850" class="external">https://en.wikipedia.org/wiki/Code_page_850</a>. So with the current subject, this bug should actually be rejected.</p>
<p>Then the question is where the U+2019 is coming from. It's rather easy to get one into an otherwise ASCII text file, e.g. with "smart quotes" or some such. The bug is therefore either in the gem (why does a gem called 'asciidoctor' use non-ascii characters :-?), in the RubyGems code, or in the win32/registry code.</p> Ruby master - Bug #10300: Encoding error in conversion from UTF-16LE to UTF-8 to CP850https://redmine.ruby-lang.org/issues/10300?journal_id=491222014-09-28T02:12:03Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Feedback</i></li></ul><p>Or from <code>FormatMessage</code>?</p>
<p>Can you try with this patch?</p>
<pre><code class="patch syntaxhl" data-language="patch"><span class="gh">index 74cc77d..4df59a9 100644
</span><span class="gd">--- a/ext/win32/lib/win32/registry.rb
</span><span class="gi">+++ b/ext/win32/lib/win32/registry.rb
</span><span class="p">@@ -174,8 +174,15 @@</span> For detail, see the MSDN[http://msdn.microsoft.com/library/en-us/sysinfo/base/pr
def initialize(code)
@code = code
msg = WCHAR_NUL * 1024
<span class="gd">- len = FormatMessageW.call(0x1200, 0, code, 0, msg, 1024, 0)
- msg = msg[0, len].encode(LOCALE)
</span><span class="gi">+ lang = 0
+ begin
+ len = FormatMessageW.call(0x1200, 0, code, lang, msg, 1024, 0)
+ msg = msg[0, len].encode(LOCALE)
+ rescue EncodingError
+ raise unless lang == 0
+ lang = 0x0409 # en_US
+ retry
+ end
</span> super msg.tr("\r".encode(msg.encoding), '').chomp
end
attr_reader :code
</code></pre> Ruby master - Bug #10300: Encoding error in conversion from UTF-16LE to UTF-8 to CP850https://redmine.ruby-lang.org/issues/10300?journal_id=491372014-09-29T12:06:58Znanarth (Adrien Bernhardt)nanarth@gmail.com
<ul></ul><p>Hello,</p>
<p>I just experienced the same problem than Guillaume on Windows 7 and tried your patch. It solved the problem perfectly.</p> Ruby master - Bug #10300: Encoding error in conversion from UTF-16LE to UTF-8 to CP850https://redmine.ruby-lang.org/issues/10300?journal_id=491392014-09-29T17:08:55Znaruse (Yui NARUSE)naruse@airemix.jp
<ul><li><strong>Target version</strong> set to <i>2.2.0</i></li></ul> Ruby master - Bug #10300: Encoding error in conversion from UTF-16LE to UTF-8 to CP850https://redmine.ruby-lang.org/issues/10300?journal_id=499192014-11-12T21:07:58Zggrossetie (Guillaume GROSSETIE)ggrossetie@gmail.com
<ul></ul><p>Sorry, I just saw your replies. I will try the patch this week and let you know.</p> Ruby master - Bug #10300: Encoding error in conversion from UTF-16LE to UTF-8 to CP850https://redmine.ruby-lang.org/issues/10300?journal_id=500192014-11-20T08:19:59Zggrossetie (Guillaume GROSSETIE)ggrossetie@gmail.com
<ul></ul><p>Nobuyoshi Nakada wrote:</p>
<blockquote>
<p>Or from <code>FormatMessage</code>?</p>
<p>Can you try with this patch?</p>
<pre><code class="patch syntaxhl" data-language="patch"><span class="gh">index 74cc77d..4df59a9 100644
</span><span class="gd">--- a/ext/win32/lib/win32/registry.rb
</span><span class="gi">+++ b/ext/win32/lib/win32/registry.rb
</span><span class="p">@@ -174,8 +174,15 @@</span> For detail, see the MSDN[http://msdn.microsoft.com/library/en-us/sysinfo/base/pr
def initialize(code)
@code = code
msg = WCHAR_NUL * 1024
<span class="gd">- len = FormatMessageW.call(0x1200, 0, code, 0, msg, 1024, 0)
- msg = msg[0, len].encode(LOCALE)
</span><span class="gi">+ lang = 0
+ begin
+ len = FormatMessageW.call(0x1200, 0, code, lang, msg, 1024, 0)
+ msg = msg[0, len].encode(LOCALE)
+ rescue EncodingError
+ raise unless lang == 0
+ lang = 0x0409 # en_US
+ retry
+ end
</span> super msg.tr("\r".encode(msg.encoding), '').chomp
end
attr_reader :code
</code></pre>
</blockquote>
<p>Thanks that solved the issue!</p> Ruby master - Bug #10300: Encoding error in conversion from UTF-16LE to UTF-8 to CP850https://redmine.ruby-lang.org/issues/10300?journal_id=505792014-12-23T07:03:52Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>Closed</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li></ul><p>Applied in changeset r48927.</p>
<hr>
<p>registry.rb: try en_US message</p>
<ul>
<li>ext/win32/lib/win32/registry.rb (Win32::Registry::Error#initialize):<br>
try en_US message if the default message cannot be encoded to<br>
locale. <a href="/issues/10300">[ruby-core:65295]</a> [Bug <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: Encoding error in conversion from UTF-16LE to UTF-8 to CP850 (Closed)" href="https://redmine.ruby-lang.org/issues/10300">#10300</a>]</li>
</ul> Ruby master - Bug #10300: Encoding error in conversion from UTF-16LE to UTF-8 to CP850https://redmine.ruby-lang.org/issues/10300?journal_id=505842014-12-23T13:34:01Zluislavena (Luis Lavena)luislavena@gmail.com
<ul><li><strong>Backport</strong> changed from <i>2.0.0: UNKNOWN, 2.1: UNKNOWN</i> to <i>2.0.0: REQUIRED, 2.1: REQUIRED</i></li></ul> Ruby master - Bug #10300: Encoding error in conversion from UTF-16LE to UTF-8 to CP850https://redmine.ruby-lang.org/issues/10300?journal_id=509932015-01-14T07:22:04Zusa (Usaku NAKAMURA)usa@garbagecollect.jp
<ul><li><strong>Backport</strong> changed from <i>2.0.0: REQUIRED, 2.1: REQUIRED</i> to <i>2.0.0: DONTNEED, 2.1: REQUIRED</i></li></ul>