Bug #6266

encoding related exception with recent integrated psych

Added by jonforums (Jon Forums) over 8 years ago. Updated almost 8 years ago.

Target version:
ruby -v:
ruby 1.9.3p176 (2012-04-04 revision 35238) [i386-mingw32]


Newer versions of Psych integrated into both ruby_1_9_3 and trunk create exceptions if the yaml file isn't perceived to be correctly encoded.

While I discovered this in my code that uses the 'logging' gem on Win7 32bit, this test snippet shows the same failure on both ruby_1_9_3@r35238 and trunk@r35229:

# hello.yml uses LF-only line endings
C:\temp>type hello.yml
:greeting: hello yaml

# yaml_test.rb uses LF-only
C:\temp>type yaml_test.rb
require 'yaml'
puts "YAML engine: #{YAML::ENGINE.yamler} #{'v'+Psych::VERSION if defined?(Psych)}"

# fix: 'rb'
# fix: 'r:utf-8'
# fix: Encoding.default_external = Encoding::UTF_8 if defined?(Encoding)
# fix: chcp 65001 in cmd.exe before running script
# fix: run script with ruby -Eutf-8'hello.yml', 'r') do |f|
puts YAML.load_stream(f)

# IBM437 codepage is a typical default for the cmd.exe shell
Active code page: 437

C:\temp>ruby -v yaml_test.rb
ruby 1.9.3p176 (2012-04-04 revision 35238) [i386-mingw32]
YAML engine: psych v1.3.1
C:/ruby193/lib/ruby/1.9.1/psych.rb:206:in parse': YAML file must be UTF-8, UTF-16LE, or UTF-16BE, not IBM437 (ArgumentError)
from C:/ruby193/lib/ruby/1.9.1/psych.rb:206:in
from C:/ruby193/lib/ruby/1.9.1/psych.rb:289:in load_stream'
from yaml_test.rb:10:in
block in '
from yaml_test.rb:9:in open'
from yaml_test.rb:9:in

# older versions of integrated psych do not cause exceptions
Active code page: 437

C:\temp>ruby -v yaml_test.rb
tcs-ruby 1.9.3p134 (2012-02-19, TCS patched 2012-02-25) [i386-mingw32]
YAML engine: psych v1.2.2
{:greeting=>"hello yaml"}

I've not tried on Arch with bash.

From the test_yaml.rb comments you can see some of the workarounds. None of these are good options because they require changes to either a users code/yaml, gem code that uses the psych library, or the ruby invocation command line.

For both trunk and ruby_1_9_3, I'd prefer psych implement a fallback strategy that Just Works combined with an informative warning rather than a hard exception.


noname (500 Bytes) noname Anonymous, 04/07/2012 10:29 AM

Also available in: Atom PDF