[EMAIL PROTECTED] wrote:
Steve Graham <[EMAIL PROTECTED]> wrote:
Okay, I've applied the following function to every number and sum and I
still get phony results. More ideas?
procedure fixNum(aNum)
i := integer((aNum * 100) + .5)
j := real(i)/100
return j
end
>
It is perfectly normal for this sort of thing to happen with floating
point numbers; it is an inherent flaw in the representation
...
You get this because the floating point numbers are in binary instead
of decimal.
Binary floating point is designed for science and engineering, not
financial calculation, which is unfortunate in this case. My guess is
you will have to work around it.
Just an additional comment. Fixed-point arithmetic is surprisingly
difficult on binary computers (where's the IBM 750 when you need it?),
as Barry points out. For example, 0.1 base 10 is a *repeating fraction*
in binary.
This would be an interesting class to write - you'd probably want to
separate the integral and fractional parts of the number out and use
integers to represent both groups (money, could be stored
as dollars and cents [or, better, dollars and mils] in those instances
where money is measured that way).
Once (if?) operator overloading is installed in Unicon, then this class
could be used quite nicely to perform fix-point operations.
--
--
Steve Wampler [EMAIL PROTECTED]
The gods that smiled upon your birth are laughing now. -- fortune cookie
-------------------------------------------------------
This SF.net email is sponsored by: 2005 Windows Mobile Application Contest
Submit applications for Windows Mobile(tm)-based Pocket PCs or Smartphones
for the chance to win $25,000 and application distribution. Enter today at
http://ads.osdn.com/?ad_id=6882&alloc_id=15148&op=click
_______________________________________________
Unicon-group mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/unicon-group
