Never mind, this is in the context of FloatingDecimal and any trailing zeroes are properly discarded. Carry on, this code looks correct, despite my misunderstanding the explanation.
David On 2013-09-12, at 12:32 PM, David Chase <david.r.ch...@oracle.com> wrote: > I think the reason you use 1075 digits is because it takes 1075 decimal > digits behind the decimal point to exactly represent 2^-1075, > and that is 1/2 ulp. When you discard the tail, if it happens to be all > zeroes, I hope you replace it with a zero, not a one, because otherwise you > can round incorrectly if the answer is otherwise equal to EVEN + 1/2 ULP. > EVEN + 1/2 ULP rounds to EVEN, EVEN + 1/2 ULP + "1" rounds to EVEN + 1. > > Whoops, no, you don't do that. But I'd call that another bug, and not one > worth getting terribly excited about. Strictly speaking, if your input is > decimal_rep_of_half_ulp + a billion zeroes + "1", then the answer is 1 ulp > (2^-1074), if the trailing one is missing, then the answer is zero (which is > even in the mantissa). > > David > > On 2013-09-12, at 10:37 AM, David Chase <david.r.ch...@oracle.com> wrote: >> On 2013-09-12, at 8:18 AM, Dmitry Nadezhin <dmitry.nadez...@gmail.com> wrote: >> >>> The reason while binary and decimal digits are mixed can be ullustrated by >>> such example. >>> It is necessary 3 decimal fraction digits to represent exactly pbinary >>> power pow(2,-3) : >>> pow(2,-3)=1/8=0.125 >>> >>> >>> On Thu, Sep 12, 2013 at 3:57 PM, David Chase >>> <david.r.ch...@oracle.com>wrote: >>> >>>> This explanation seems to combine numbers of binary digits (1075) >>>> and numbers of decimal digits (17), and therefore makes me a little >>>> nervous, though I think 1100 is a conservative choice that, even if not >>>> 100% correct, will be 99.(over 700 9s)% correct. >>>> >>>> David >> >> Yes, but you are adding them. It is as if you took two lengths, 1075 feet >> and 17 yards, added them, rounded up, and said that 1100 yards will be >> adequate (which is entirely true, though perhaps overconservative). >> Overconservative is okay. >> >> 2^-1075 is the binary ulp, give or take a fencepost, not the decimal ulp. >> >> David >> >
