Project

General

Profile

Actions

Bug #8445

open

IO.open and IO#set_enconding does not support :fallback option

Added by pjmtdw (Haruhiro Yoshimoto) almost 11 years ago. Updated 4 months ago.

Status:
Assigned
Target version:
-
ruby -v:
trunk(ruby 2.1.0dev)
Backport:
[ruby-dev:47369]
Tags:

Description

RubyDoc says that IO.open and IO#set_encoding supports optional argument defined in String#encode.
http://ruby-doc.org/core-2.0/IO.html#method-c-new-label-Options
In fact, :invalid, :undef and :replace works as expected.

However, :fallback option does not work neither for IO.open and IO#set_encoding.
Following is the example code which does not work.
f(x) is never called even if hoge.txt contains non convertible character.

File.open("./hoge.txt","r:Shift_JIS:utf-8", :fallback => lambda{|x|f(x)}){|f|
  ...
}
File.open("./hoge.txt"){|f|
  f.set_encoding("Shift_JIS","utf-8",:fallback => lambda{|x|f(x)})
  ...
}

I Think this is because fill_cbuf() in io.c calls rb_econv_convert() from transcode.c directly.
On the other hand, fallback_func is called in transcode_loop(), which is called by str_encode().

Since transcode_loop() also calls rb_econv_convert(), I wrote a small patch which moves some codes from
transcode_loop() to rb_econv_convert() to fix the problem.

The attached file is the patch. Hope this helps.


Files

support-fallback-for-io.patch (3.9 KB) support-fallback-for-io.patch pjmtdw (Haruhiro Yoshimoto), 05/24/2013 10:03 PM

Updated by naruse (Yui NARUSE) almost 11 years ago

  • Category set to M17N
  • Status changed from Open to Assigned
  • Assignee set to akr (Akira Tanaka)
  • Target version set to 2.1.0

Is this intentional or just unexpected inconsistency?

Updated by hsbt (Hiroshi SHIBATA) about 10 years ago

  • Target version changed from 2.1.0 to 2.2.0
Actions #3

Updated by naruse (Yui NARUSE) about 6 years ago

  • Target version deleted (2.2.0)
Actions #4

Updated by ko1 (Koichi Sasada) almost 2 years ago

  • Description updated (diff)
  • Backport deleted (1.9.3: UNKNOWN, 2.0.0: UNKNOWN)

Updated by JesseJohnson (Jesse Johnson) 4 months ago

I can replicate this in 3.2.2.

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0