Bug #4308: Wrong Bignum calculation in some case.

• Status: Rejected
• Assignee: -
• Target version: -
• ruby -v: ruby 1.9.2p136 (2010-12-25 revision 30365) [x86_64-darwin10.5.0]

[shouichi (Shouichi Kamiya)]

=begin
I tried to calculate variance of an array but calculation returns wrong result
when numbers are big.The problem also happens in 1.8.7-p330.

I attached a file that recreates The problem.
=end

[naruse (Yui NARUSE)]

• Status changed from Open to Rejected

=begin
It seems from floating point's accuracy.
In line 9, it uses to_f and the value exceeds its limit.
=end

[nobu (Nobuyoshi Nakada)]

=begin
#!/usr/bin/env ruby

class Array
def sum
reduce(:+)
end

def var
empty? ? 0.0 : (map {|x| x ** 2}.sum * size - sum ** 2).fdiv(size ** 2)
end
end

All of these should clearly return 0.

p [1295218865, 1295218865, 1295218865, 1295218865, 1295218865].var
#=> 0.0

p [1295218865, 1295218865, 1295218865, 1295218865, 1295218865, 1295218865].var
#=> 0.0

p [1295218865, 1295218865, 1295218865, 1295218865, 1295218865, 1295218865, 1295218865].var
#=> 0.0

=end