Mike Davis wrote:
> I have been giving this a lot of thought and I just think it's the
> fact that not all decimals can be represented to 9 decimal places,
> even within the 52 bits. As the size of the integer part increases
> more and more of these 52 bits are used up to represent the integer
> part. So for large numbers there are not enough remaining bits of
> precision to represent the decimal part.
>
> For instance: 12345678.12 gives 12345678.119999999 rounding error and
> that is within 16 digits. The worst error occurs in the 9th decimal
> spot. So, I think if I limit my NumberOfDecPlaces to 8 and the total
> number of digits to 16, that my function rounding will make the
> display look right.
This is getting a little off topic, as we're really discussing floating
point arithmetic in general, and nothing specific to the Palm, but I'll
just add a couple more thoughts.
You're essentially correct when you say that certain decimals can't be
represented to 9 decimal digits in 52 bits. The fact is, certain
decimals can't be represented exactly in ANY finite number of bits,
because they have infinite repeating representations when converted to
binary. For example, 0.1 (one tenth) in decimal is in binary,
0.00011001100110011... repeated forever. (On the other hand, certain
numbers that look like they have a complicated representation in decimal
are simpler in binary; like 1/(2^50), which has ONE DIGIT of
significance in binary representation, but many digits of significance
in decimal.) Anyway, you're not going to get an exact representation of
0.1 in ANY number of bits.
However, when you convert an N bit binary number back to decimal for
display, you should round the last digit. By "last digit" I mean
whatever is the decimal digit which corresponds to the last (N-th) bit
of significance in the binary representation. For 52 bit doubles,
that's about the 16th decimal digit. Then a number like
12345678.119999999 will be displayed as 12345678.12. You're not losing
anything by rounding at the Nth bit, because there *are* no more bits
after that. If you do decimal arithmetic (keep dividing by 10 for
example), it may look like you're getting more digits, but it's just
"noise"; they have no relation to the original number.
There's no need to limit the number of places to the left (or right) of
the decimal point, because the position of the decimal point is
irrelevant. The number is stored (as a double) as 52 bits of precision
(with no "binary point"), and 11 bits that tell where the "binary point"
is. Those are two independent parts of the number, and changing the
position of the binary point doesn't affect the number of digits of
precision.
I went through a lot of these same issues when I wrote the
double-to-ascii converter for my calculator Unc (dredging up long
forgotten memories of my Numerical Analysis class in college). You can
play around with Unc and see how it handles these things if you like.
Hope this helps.
--Mark
