Feature #16655

closed

Each test on test-all should run `srand(seed)` at setup

Added by ko1 (Koichi Sasada) almost 5 years ago. Updated about 4 years ago.

Status:

Closed

Assignee:

ko1 (Koichi Sasada)

Target version:

3.0

[ruby-core:97272]

Description

Points¶

call srand($seed) before each test. $seed is given seed number by --seed N

Background¶

Some tests use rand() to produce random number to generate random test examples.

Example:

  def test_local_variable_set_wb
    assert_ruby_status([], <<-'end;', '[Bug #13605]', timeout: 30)
      b = binding
      n = 20_000

      n.times do |i|
        v = rand(2_000)
        name = "n#{v}"
        value = Object.new
        b.local_variable_set name, value
      end
    end;
  end

This test generates 2,000 randomly named ivar which point random numbers.

However, several tests calls srand(0) and this random sequence is affected by test methods order.

Two problems:

Test authors of this kind of tests should assume that every test processes uses very different random sequence to increase the test coverage.
We can not get reproducible results on such tests which use random sequence with --seed because of srand(0) using tests.

Proposal¶

To solve the above two problems, I'll insert srand($seed) at the beginning of each test ($seed is given by --seed option).
This proposal respects --seed option more, so we can expect more deterministic results.

I measured performance overhead and I can't get meaningful slow down.