Project

General

Profile

Actions

Bug #14635

closed

Float#round(n) returns a wrong result when n is big

Added by mame (Yusuke Endoh) about 6 years ago. Updated over 2 years ago.

Status:
Closed
Assignee:
-
Target version:
-
[ruby-core:86323]
Tags:

Description

First of all, don't confuse that this is a usual floating-point error issue.

The following looks inconsistent:

3.0e-31           #=> 3.0e-31
3.0e-31.round(31) #=> 3.0000000000000003e-31

What it should be

A Float value is actually a range.

3.0e-31 represents a range of 0.299999999999999959315060e-30 .. 0.300000000000000003105637e-30 (the bounds are approximate). I call this range "A".
3.0000000000000003e-31 represents a range of 0.300000000000000003105637e-30 .. 0.300000000000000046896214e-30. I call this range "B".

x.round(31) should (1) multiple x with 10**31, (2) round it as an integer, and (3) divide it with 10**31.

In this case:

(1) 3.0e-31 * 10**31 is a range of 2.99999999999999959315060 .. 3.00000000000000003105637.
(2) The rounded result is 3, whichever value is chosen from the range above.
(3) 3.0 / 10**31 is within the range "A", not within the range "B", so the result should be 3.0e-31, not 3.0000000000000003e-31.

How the bug occurs

The reason why 3.0e-31.round(31) returns 3.0000000000000003e-31, is the implementation issue of Float#round. It does the following:

(1) f = pow(10, b)
(2) x = round(x * f) as an integer
(3) return x / f

However, a double variable f cannot represent pow(10, 31) precisely. In other words, the 10**31 must be handled as an integer, but the implementation handles it as an inexact floating-point value. This is the issue.

How to fix

The issue is simple, but it might be very difficult to fix. strtod handles a string "3.0e-31" correctly. So, by doing the same as strtod, this issue would be fixed. However, the strtod implementation looks very difficult, at least to me. Contribution from mathematician is welcome.
(Honestly, I don't want to see such a complication in the source code. Another simpler approach would be more preferable.)

References

This issue has been already reported in #5273 by marcandre. But the status of the ticket looks unclear; I cannot see how many issues remains. So, I created this ticket for just one bug that I could confirm.


Related issues 2 (0 open2 closed)

Related to Ruby master - Bug #5273: Float#round returns the wrong floats for higher precisionClosedmarcandre (Marc-Andre Lafortune)Actions
Related to Ruby master - Bug #17183: Float round working weirdlyClosedActions
Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0