Project

General

Profile

Actions

Feature #16655

closed

Each test on test-all should run `srand(seed)` at setup

Added by ko1 (Koichi Sasada) about 4 years ago. Updated over 3 years ago.

Status:
Closed
Target version:
[ruby-core:97272]

Description

Points

  • call srand($seed) before each test. $seed is given seed number by --seed N

Background

Some tests use rand() to produce random number to generate random test examples.

Example:

  def test_local_variable_set_wb
    assert_ruby_status([], <<-'end;', '[Bug #13605]', timeout: 30)
      b = binding
      n = 20_000

      n.times do |i|
        v = rand(2_000)
        name = "n#{v}"
        value = Object.new
        b.local_variable_set name, value
      end
    end;
  end

This test generates 2,000 randomly named ivar which point random numbers.

However, several tests calls srand(0) and this random sequence is affected by test methods order.

Two problems:

  • Test authors of this kind of tests should assume that every test processes uses very different random sequence to increase the test coverage.
  • We can not get reproducible results on such tests which use random sequence with --seed because of srand(0) using tests.

Proposal

To solve the above two problems, I'll insert srand($seed) at the beginning of each test ($seed is given by --seed option).
This proposal respects --seed option more, so we can expect more deterministic results.

I measured performance overhead and I can't get meaningful slow down.

Additional note

Surprisingly, --seed option is ignored just now on test-all, so I'll fix this problem too.

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0