On 11/9/05, Andrew Lentvorski <[EMAIL PROTECTED]> wrote:
> [EMAIL PROTECTED] wrote:
> > IMO, a more detailed answer would help.
>
> Okay, I'll try to do that.
>
> > There is a large body of knowledge about this issue. Unfortunately
> > I'm not informed enough to go much beyond naming some of the issues.
>
> This is probably the best site to that end:
> http://www2.hursley.ibm.com/decimal/
>
> This is the FAQ surrounding why Decimal arithmetic is important:
> http://www2.hursley.ibm.com/decimal/decifaq.html
>
> Quoting from the FAQ:
> > 2. Even a single operation can give very unexpected results. For
> > example:
> >
> > * Consider the calculation of a 5% sales tax on an item (such as a
> > $0.70 telephone call), which is then rounded to the nearest cent.
> >
> > Using double binary floating-point, the result of 0.70 x 1.05 is
> > 0.73499999999999998667732370449812151491641998291015625; the result
> > should have been 0.735 (which would be rounded up to $0.74) but
> > instead the rounded result would be $0.73.
> >
> > Similarly, the result of 1.30 x 1.05 using binary is
> > 1.3650000000000002131628207280300557613372802734375; this would be
> > rounded up to $1.37. However, the result should have been 1.365 –
> > which would be rounded down to $1.36 (using Banker's rounding).
> >
> > Taken over a million transactions of this kind, as in the 'telco'
> > benchmark, these systematic errors add up to an overcharge of more
> > than $20. For a large company, the million calls might be
> > two-minutes-worth; over a whole year the error then exceeds $5
> > million.
Aren't there just as many cases in which the
decimal-to-binary-to-decimal errors undercharge rather than
overcharge?
> > 3. Binary calculations can make apparently predictable decisions
> > unsafe. For example, the Java loop:
> >
> > for (double d = 0.1; d <= 0.5; d += 0.1) System.out.println(d);
> >
> >
> > displays five numbers, whereas the similar loop:
> >
> > for (double d = 1.1; d <= 1.5; d += 0.1) System.out.println(d);
> >
> >
> > displays only four numbers. (If d had a decimal type then five
> > numbers would be displayed in both cases.)
> >
I think I learned this while writing my first Fortran program, circa 1959.
Count with integers, compute with floats.
carl
--
carl lowenstein marine physical lab u.c. san diego
[EMAIL PROTECTED]
--
[email protected]
http://www.kernel-panic.org/cgi-bin/mailman/listinfo/kplug-list
