I think the reason you use 1075 digits is because it takes 1075 decimal digits behind the decimal point to exactly represent 2^-1075, and that is 1/2 ulp. When you discard the tail, if it happens to be all zeroes, I hope you replace it with a zero, not a one, because otherwise you can round incorrectly if the answer is otherwise equal to EVEN + 1/2 ULP. EVEN + 1/2 ULP rounds to EVEN, EVEN + 1/2 ULP + "1" rounds to EVEN + 1.
Whoops, no, you don't do that. But I'd call that another bug, and not one worth getting terribly excited about. Strictly speaking, if your input is decimal_rep_of_half_ulp + a billion zeroes + "1", then the answer is 1 ulp (2^-1074), if the trailing one is missing, then the answer is zero (which is even in the mantissa). David On 2013-09-12, at 10:37 AM, David Chase <david.r.ch...@oracle.com> wrote: > On 2013-09-12, at 8:18 AM, Dmitry Nadezhin <dmitry.nadez...@gmail.com> wrote: > >> The reason while binary and decimal digits are mixed can be ullustrated by >> such example. >> It is necessary 3 decimal fraction digits to represent exactly pbinary >> power pow(2,-3) : >> pow(2,-3)=1/8=0.125 >> >> >> On Thu, Sep 12, 2013 at 3:57 PM, David Chase <david.r.ch...@oracle.com>wrote: >> >>> This explanation seems to combine numbers of binary digits (1075) >>> and numbers of decimal digits (17), and therefore makes me a little >>> nervous, though I think 1100 is a conservative choice that, even if not >>> 100% correct, will be 99.(over 700 9s)% correct. >>> >>> David > > Yes, but you are adding them. It is as if you took two lengths, 1075 feet > and 17 yards, added them, rounded up, and said that 1100 yards will be > adequate (which is entirely true, though perhaps overconservative). > Overconservative is okay. > > 2^-1075 is the binary ulp, give or take a fencepost, not the decimal ulp. > > David >
