Bug #6673: No longer able to run tests in parallel (-j2)

• Status: Closed
• Assignee: sorah (Sorah Fukumori)
• Target version: 2.0.0
• ruby -v: ruby 2.0.0dev [i386-mingw32] at r36254

[luislavena (Luis Lavena)]

=begin
Hello,

Since a few revisions back, I'm no longer able to run tests in parallel, getting output like the following:

Running tests:

[666/690] 116=test_utils 4824=test_err_in_callbackC:/Users/Worker/Code/ruby/ruby/lib/mkmf.rb:75:in split': invalid byte sequence in UTF-8 (ArgumentError) from C:/Users/Worker/Code/ruby/ruby/lib/mkmf.rb:75:in block in module:MakeMakefile'
from C:/Users/Worker/Code/ruby/ruby/lib/mkmf.rb:74:in each' from C:/Users/Worker/Code/ruby/ruby/lib/mkmf.rb:74:in module:MakeMakefile'
from C:/Users/Worker/Code/ruby/ruby/lib/mkmf.rb:42:in <top (required)>' from C:/Users/Worker/Code/ruby/ruby/lib/rubygems/custom_require.rb:36:in require'
from C:/Users/Worker/Code/ruby/ruby/lib/rubygems/custom_require.rb:36:in require' from C:/Users/Worker/Code/ruby/ruby/test/win32ole/test_err_in_callback.rb:11:in <top (required)>'
from C:/Users/Worker/Code/ruby/ruby/lib/rubygems/custom_require.rb:36:in require' from C:/Users/Worker/Code/ruby/ruby/lib/rubygems/custom_require.rb:36:in require'
from C:/Users/Worker/Code/ruby/ruby/lib/test/unit/parallel.rb:114:in run' from C:/Users/Worker/Code/ruby/ruby/lib/test/unit/parallel.rb:177:in '
Some worker was crashed. It seems ruby interpreter's bug
or, a bug of test/unit/parallel.rb. try again without -j
option.

This was working previously, even on Windows (the platform I'm reporting right now)

Base Ruby is 1.9.3-p194 [i386-mingw32]
Target Ruby is ruby 2.0.0dev (2012-06-29) [i386-mingw32] at r36254

=end

[sorah (Sorah Fukumori)]

I can't reproduce this on OS X, is this still reproducable?

If you can still reproduce this on trunk, I'll prepare mingw environment on my Windows box.

[sorah (Sorah Fukumori)]

• Status changed from Open to Feedback
• Assignee set to sorah (Sorah Fukumori)

[luislavena (Luis Lavena)]

sorah (Shota Fukumori) wrote:

I can't reproduce this on OS X, is this still reproducable?

Yes, this still reproduces on trunk (r36393)

If you can still reproduce this on trunk, I'll prepare mingw environment on my Windows box.

Perhaps you can take a look to RubyChallengePack I prepared, which includes a base Ruby, binaries used by RubyInstaller, GCC + MSYS for building.

You only need Git and a clone of Ruby source.

Details here:

https://groups.google.com/d/topic/rubyinstaller/A92K_EYJX2A/discussion

One note about cloning: ensure Git autocrlf is false:

git config --global core.autocrlf false

So sed works properly with configure script (cloning with CRLF generates some issues).

[sorah (Sorah Fukumori)]

• Status changed from Feedback to Assigned

Thanks, I'll take a look for this, but it may take long time to start investigating. sorry!

[h.shirosaki (Hiroshi Shirosaki)]

=begin
I was able to reproduce this with mingw ruby.
ruby 2.0.0dev (2012-09-24 trunk 37025) [x64-mingw32]

If test_gem_installer.rb and test_err_in_callback.rb run in same process, this error seems to occur.

$ make test-all TESTS="-q -j2 drb rubygems/test_gem_installer.rb win32ole/test_err_in_callback.rb"

[6/6] 4324=test_drbssl 4340=test_err_in_callbackc:/Users/hiroshi/work/ruby/lib/mkmf.rb:80:in split' : invalid byte sequence in UTF-8 (ArgumentError) c:/Users/hiroshi/work/ruby/lib/mkmf.rb:80:in block in module:MakeMakefile'
c:/Users/hiroshi/work/ruby/lib/mkmf.rb:79:in each' c:/Users/hiroshi/work/ruby/lib/mkmf.rb:79:in module:MakeMakefile'
c:/Users/hiroshi/work/ruby/lib/mkmf.rb:46:in <top (required)>' c:/Users/hiroshi/work/ruby/lib/rubygems/custom_require.rb:36:in require'
c:/Users/hiroshi/work/ruby/lib/rubygems/custom_require.rb:36:in require' c:/Users/hiroshi/work/ruby/test/win32ole/test_err_in_callback.rb:11:in <top (required)>'
c:/Users/hiroshi/work/ruby/lib/rubygems/custom_require.rb:36:in require' c:/Users/hiroshi/work/ruby/lib/rubygems/custom_require.rb:36:in require'
c:/Users/hiroshi/work/ruby/lib/test/unit/parallel.rb:114:in run' c:/Users/hiroshi/work/ruby/lib/test/unit/parallel.rb:181:in '

Some worker was crashed. It seems ruby interpreter's bug
or, a bug of test/unit/parallel.rb. try again without -j
option.

test_gem_installer.rb sets invalid encoding string to ARGV. To clear ARGV before running test is a possible fix.

diff --git a/lib/test/unit/parallel.rb b/lib/test/unit/parallel.rb
index 921c6b0..7546630 100644
--- a/lib/test/unit/parallel.rb
+++ b/lib/test/unit/parallel.rb
@@ -110,6 +110,7 @@ module Test
@options = @opts.dup
suites = MiniTest::Unit::TestCase.test_suites

•          ARGV.clear
           begin
             require $1
           rescue LoadError
  

=end

[nobu (Nobuyoshi Nakada)]

• Status changed from Assigned to Closed
• % Done changed from 0 to 100

This issue was solved with changeset r37032.
Luis, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

---

test_gem_installer.rb: restore ARGV

• test/rubygems/test_gem_installer.rb
  (TestGemInstaller#test_install_creates_binstub_that_dont_trust_encoding):
  restore ARGV properly, non-ascii strings in different encodings are
  not equal. [ruby-core:45975] [Bug #6673]