[ https://issues.apache.org/jira/browse/NUMBERS-120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Heinrich Bohne resolved NUMBERS-120. ------------------------------------ Resolution: Fixed Fix Version/s: 1.0 > Major loss of precision in BigFraction.doubleValue() and > BigFraction.floatValue() > --------------------------------------------------------------------------------- > > Key: NUMBERS-120 > URL: https://issues.apache.org/jira/browse/NUMBERS-120 > Project: Commons Numbers > Issue Type: Bug > Components: fraction > Affects Versions: 1.0 > Reporter: Heinrich Bohne > Priority: Minor > Fix For: 1.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > The method {{BigFraction.doubleValue()}} calculates the double value of > fractions with numerators or denominators that, when converted to a > {{double}}, round up to {{Double.POSITIVE_INFINITY}}, by right-shifting both > the numerator and denominator synchronously until both numbers fit into 1023 > bits. Apart from the fact that the maximum number of bits an integer > representable as a finite {{double}} can have is 1024 (an unbiased exponent > of 1023, which is the largest possible unbiased exponent of a {{double}} > number, means 1.xxxx ⋅ 2^1023^, which amounts to 1024 bits), this way of > converting the fraction to a {{double}} is incredibly wasteful with precision > if the numerator and denominator have a different bit length, because the > smaller of the two numbers will be truncated beyond what is necessary to > represent it as a finite {{double}}. Here is an extreme example: > The smallest integer that rounds up to {{Double.POSITIVE_INFINITY}} when > converted to a {{double}} is 2^1024^ - 2^970^. This is because > {{Double.MAX_VALUE}} as an integer is a 1024-bit number with the most > significant 53 bits set to 1 and all other 971 bits set to 0. If the 970 > least significant bits are changed in any way, the number will still round > down to {{Double.MAX_VALUE}} as long as the 971st bit remains 0, but as soon > as the 971st bit is set to 1, the number will round up to > {{Double.POSITIVE_INFINITY}}. > The smallest possible denominator greater than 1 where a single right-shift > will cause a loss of precision is 3. 2^1024^ - 2^970^ is divisible by 3, so > in order to create an irreducible fraction, let's add 1 to it: > (2^1024^ - 2^970^ + 1) / 3 ≈ 5.992310449541053 ⋅ 10^307^ (which can be > verified with {{BigDecimal}}, or, more easily, with [this online > tool|https://www.wolframalpha.com/input/?i=(2%5E1024+-+2%5E970+%2B+1)+%2F+3]. > However, the current implementation of BigFraction.doubleValue() returns > 8.98846567431158 ⋅ 10^307^, which differs from the correct result by a > relative error of 50%! The same problem applies to the method > {{BigFraction.floatValue()}}. > This can be prevented by truncating the numerator and denominator separately, > so that for each of the two numbers, the maximum possible precision is > retained. -- This message was sent by Atlassian JIRA (v7.6.14#76016)