Bug #6765: test_filename_utf8_raw_name and test_filename_euc_jp (test_dir_m17n.rb) fail on non-JapaneseWindows

• Status: Closed
• Assignee: luislavena (Luis Lavena)
• Target version: 2.0.0
• ruby -v: ruby 2.0.0dev (2012-07-21) [i386-mingw32]

[os97673 (Oleg Sukhodolsky)]

It looks like the problem is related to http://bugs.ruby-lang.org/issues/4799
The problem is that these test use "\u3042" as a file name to create and to test its name in ASCII-8BIT mode.
But it expects "\x82\x40" as a result, but this is "\u3042".encode('sjis') ('jsis' is a filesystem encoding
on Japanese Windows).
To fix the test I do changed it to calculate expected file name using original name and filesystem's encoding.
Also, I've added two more tests with Windows-1251 and Windows-1252 symbols as file name.
And they does demonstrated another problem of the test on Windows: result of the function Ruby uses to
get binary version of filename (WideCharToMultiByte) is not deterministric (see
http://msdn.microsoft.com/en-us/library/windows/desktop/dd374047(v=vs.85).aspx#SC_char_conv_func for more details).
This is why I've added additional check to which test to check if at least we have expected file name in case we use fylesystem's encoding.

[luislavena (Luis Lavena)]

• Status changed from Open to Feedback

=begin
Thank you Oleg for the patch.

I just tested it, but still have a failure:

Before patch:

Running tests:

[ 4/10] TestDir_M17N#test_filename_euc_jp = 0.08 s

1. Failure:
   test_filename_euc_jp(TestDir_M17N) [C:/Users/Worker/Code/ruby/ruby/test/ruby/test_dir_m17n.rb:170]:
   ruby exit status is not success: #<Process::Status: pid 4500 exit 1>

[ 9/10] TestDir_M17N#test_filename_utf8_raw_name = 0.08 s
2) Failure:
test_filename_utf8_raw_name(TestDir_M17N) [C:/Users/Worker/Code/ruby/ruby/test/ruby/test_dir_m17n.rb:190]:
ruby exit status is not success: #<Process::Status: pid 6892 exit 1>

Finished tests in 0.657084s, 15.2188 tests/s, 27.3938 assertions/s.
10 tests, 18 assertions, 2 failures, 0 errors, 0 skips

ruby -v: ruby 2.0.0dev (2012-07-21 trunk 36486) [i386-mingw32]

After patch:

Running tests:

[ 4/12] TestDir_M17N#test_filename_euc_jp = 0.08 s

1. Failure:
   test_filename_euc_jp(TestDir_M17N) [C:/Users/Worker/Code/ruby/ruby/test/ruby/test_dir_m17n.rb:196]:
   ruby exit status is not success: #<Process::Status: pid 6596 exit 1>

Finished tests in 0.809603s, 14.8221 tests/s, 27.1738 assertions/s.
12 tests, 22 assertions, 1 failures, 0 errors, 0 skips

ruby -v: ruby 2.0.0dev (2012-07-21 trunk 36486) [i386-mingw32]

Seems like (({test_filename_euc_jp})) is still failing.

My codepage is 1252, changing it to 1251 seems to solve the issue, which indicate some minor issue with this test.

Can you confirm with chcp 1252?

Thank you.
=end

[os97673 (Oleg Sukhodolsky)]

• File test_dir_m17n.rb.v2.diff test_dir_m17n.rb.v2.diff added

The patch updated to make test_filename_euc_jp pass on Windows-1252.

[luislavena (Luis Lavena)]

• Status changed from Feedback to Assigned
• Target version set to 2.0.0

=begin
Thank you Oleg,

I can confirm new patch works:

Running tests:

Finished tests in 0.785099s, 15.2847 tests/s, 28.0219 assertions/s.
12 tests, 22 assertions, 0 failures, 0 errors, 0 skips

ruby -v: ruby 2.0.0dev (2012-07-21 trunk 36486) [i386-mingw32]

Nobu, Usa, can I commit?

Any with Japanese-Windows to double check?

Thank you
=end

[luislavena (Luis Lavena)]

• Status changed from Assigned to Closed
• % Done changed from 0 to 100

This issue was solved with changeset r36509.
Oleg, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

---

Fix raw UTF-8 test for non-Japanase Windows systems

• test/ruby/test_dir_m17n.rb (create_and_check_raw_file_name): add new
  helper method to ease encoding testing. Patch by Oleg Sukhodolsky.
  [ruby-core:46589][Bug #6765]

• test/ruby/test_dir_m17n.rb (test_filename_extutf8): use filesystem
  encoding when reading entries and comparing.

• test/ruby/test_dir_m17n.rb (test_filename_utf8_raw_name): removed.

• test/ruby/test_dir_m17n.rb (test_filename_utf8_raw_jp_name): split test.