This explanation seems to combine numbers of binary digits (1075) and numbers of decimal digits (17), and therefore makes me a little nervous, though I think 1100 is a conservative choice that, even if not 100% correct, will be 99.(over 700 9s)% correct.
David On 2013-09-12, at 1:17 AM, Dmitry Nadezhin <dmitry.nadez...@gmail.com> wrote: > When I prepared the patch I didn't try to find the optimal MAX_NDIGITS, but > I was sure that MAX_NDIGITS = 1100 is enough. > Here is an expanation: > 1) If value of decimal string is larger than pow(10,309) then it is > converted to positive infinity before the test of nDigits; > 2) If value of decimal string is smaller than pow(10,309) and larger than > pow(2,53) then it rounds to Java double with ulp >= 2. > Half-ulp >= 1. > The patch is correct when decimal ULP of kept digits is 1 or less. It is > true beacuse MAX_NDIGITS = 1100 > 309; > 3) If value of decimal string is smaller than pow(2,53) but larger than > 0.5*Doulle.MIN_VALUE = pow(2,-1022-52-1), than > binary ULP is a multiple of pow(2,-1074). > Half-ULP is a multiple of pow(2,-1075). > The patch is correct when decimal ULP of kept digits is pow(10,-1075) of > less. > pow(2,53) has 17 decimal decimal digits. MAX_NDIGITS = 1100 > 1075 + 17 . > 4) If value of decimal string is smaller than 0.5*Double.MIN_VALUE then it > is converted to zero before the test of nDigits. > > > You can treat replacement of 1075 + 17 by 1100 as my paranoia. > It still was interesting for me what is the optimal truncation of decimal > string.
