Project

General

Profile

Actions
Copy link

Bug #22246

open

Rational#to.f incorrectly rounds when the numerator is a bignum that cannot be represented exactly by a double, and the denominator is a fixnum

Bug #22246: Rational#to.f incorrectly rounds when the numerator is a bignum that cannot be represented exactly by a double, and the denominator is a fixnum

Added by AndrewDile (Andrew Dile) about 4 hours ago.

Status:
Open
Assignee:
-
Target version:
-
ruby -v:
4.0.6
Backport:
3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN
[ruby-core:126395]

Description

Rational(65803600513127829623, 10**12).to_f

The bignum numerator cannot be exactly represented by a double, but the code in rb_big_fdiv_double rounds it to a double (dx) before dividing by the fixnum denominator of 10^12, resulting in 65803600.51312783, instead of the correctly rounded value, 65803600.513127826. The isinf(dx) test does not cover this case. Moreover, not all fixnums (in 64-bit systems) can be represented exactly as doubles.

No data to display

Actions
Copy link

Also available in: PDF Atom