On Wed, 21 Jan 2004 11:00:53 +0200, "dilek.ozgu" <[EMAIL PROTECTED]> wrote:
>hi. >double xx; > xx = 816 / 10 => 81.5999999 ?????????? >it must be 81.60 what is the solution? Even though 81.60 is expressible exactly using a finite number of decimal places, it is not expressible in binary using a finite number of binary fractional bits. So any "float" or "double" representation of this number is bound to be an approximation. The only question is then what form that number will take when it is formatted back into decimal form with a finite number of decimal places. If the formatting rountine you are using is any good, it will correctly round off to the nearest decimal number, given the number of decimal places you requested. If you request a small enough number of decimal places (so as not to tax the resolution of the double representation too much), then you should get 81.6000000. Floats have a resolution of about 7 decimal digits. Doubles, if they use the 8-byte format, have about twice that. There is an option in CodeWarrior where you can choose whether "double" uses 8 bytes, or is equivalent to "float" and uses only 4 bytes. If you have chosen the 4-byte doubles, then it is entirely possible that 816.0/10.0 would round off to 81.5999999. But if you have chosen 8-byte doubles, and if all relevant calculations are carried out in doubles, and if your formatting routine rounds correctly, then I would expect the %10.7f format version of this number to be 81.6000000. Robert Scott, Ypsilanti, MI (reply through this forum, not by e-mailing me directly) -- For information on using the Palm Developer Forums, or to unsubscribe, please see http://www.palmos.com/dev/support/forums/
