Project

General

Profile

Actions

Bug #9766

closed

Add force_encoding option to csv

Added by dtaniwaki (DAISUKE TANIWAKI) over 10 years ago. Updated over 10 years ago.

Status:
Closed
Target version:
ruby -v:
all
[ruby-core:62113]

Description

Hi there,

I have a trouble when I use csv#generate with encoding 'Shift-JIS' option. I investigated it for a long time and found it is caused by compatibility within "UTF-8" and "Shift-JIS". Since "Shift-JIS" can be converted to UTF-8, a row with UTF-8 strings added to the csv instance makes the encoding of whole rows UTF-8.

Take a look at the code below.
https://github.com/dtaniwaki/ruby/blob/trunk/lib/csv.rb#L1658

Here's the code example.

irb(main):002:0> s = generate(encoding: 'SJIS') do |csv|
    csv << ['あ']
  end
=> ["あ"]

irb(main):003:0> s
=> "あ\n"

irb(main):004:0> s.encoding
=> #<Encoding:UTF-8>

I was intended to make SJIS encoded csv, but the result was UTF-8 csv. I think everyone think it should generate Shift-JIS encoded csv string, so could you consider to merge the change attached to this issue?

The expected result is here.

irb(main):002:0> s = generate(encoding: 'SJIS', force_encoding: true) do |csv|
    csv << ['あ']
  end
=> ["あ"]

irb(main):003:0> s
=> "\x{E381}\x82\n"

irb(main):004:0> s.encoding
=> #<Encoding:Windows-31J>

Files

csv.rb.diff (1.41 KB) csv.rb.diff Diff from revision bdeedccc5fb9131cff58cffd3428d30117bc0e74 in trunk dtaniwaki (DAISUKE TANIWAKI), 04/21/2014 07:36 AM
0001-csv.rb-honor-encoding-option.patch (2.64 KB) 0001-csv.rb-honor-encoding-option.patch nobu (Nobuyoshi Nakada), 04/22/2014 03:54 AM
Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0