Jules,
For financial applications, it's far more problematic than testing for equality.
For example, if you are applying a discount to a price the resulting price must
have exact two digit decimal accuracy, otherwise by the time you have
discounted a few items you have an invoice whose total doesn't match the sum of
the discounted items as shown on the invoice. That tends to upset customers.
The other thing to consider is that such arithmetic must be consistent on
server as well as client, otherwise you can end up with server-side reports
that don't match client-side displays.
Paul
----- Original Message -----
From: Jules Suggate
To: flexcoders@yahoogroups.com
Sent: Wednesday, December 17, 2008 4:36 PM
Subject: Re: [flexcoders] Rounding error
Valdhor already gave you all the info you need, but I like the sound of my
own fingers typing. I also like to procrastinate, so I'm in heaven right now :p
a.. Some numbers that are easy to represent in decimal are hard to
represent in binary, just as some numbers that are easy to represent in base 60
are hard to represent in decimal (such as 1/3).
b.. Your calculations are safe enough: Number has a very high precision and
calculations are carried out with full precision. It's just when testing for
equality that you can run into problems -- I'm guessing this is where you
noticed the discrepancy?
c.. When comparing for floating-point numbers for equality, you need to
specify a tolerance too. Most unit-testing frameworks provide this for their
floating-point asserts. And when I was doing financial apps, we did the same
thing. If you *really* need higher precision, you'll need a BigNumber
implementation for AS3, of which there are several. The AS3Crypto library has
one I believe...
Chyaaaars :)
Jules
On Thu, Dec 18, 2008 at 03:15, Ken Johnson <kenjohnso...@hotmail.com> wrote:
Hi Everyone -
Given the following code:
var num1:Number = 0.72;
var num2:Number = 0.198;
var num3:Number;
num3 = num1;
num3 += num2;
I would expect num3 to equal 0.918.
Instead, it equals 0.9179999999999999.
And when I apply a number formatter with a precision of 2, I get 0.91.
I am performing financial calculations, and this is not acceptable.
Am I doing something wrong here?
Thanks for your help!
Ken
--
Cheers,
Jules
--
Jules Suggate
Owner and Technical Lead
Uphill Sprint Limited
+64-21-157-8562
