That's pretty much it.
To a computer Math.cos(Math.PI/2) is not 0. It's really close to 0,
because PI is an infinite sequence and a computer can "only" store it
as a double precision floating point number (ie, a fixed value).
What you get back from this calculation is the error bound of the
computer basically, which you can then use for numerical
calculations, ie, MathLib.ERROR_BOUND = Math.cos(Math.PI/2). Then you
can feasibly use if you need numerical accuracy.
IE, if result == MathLib.ERROR_BOUND, result = 0.
Numerical accuracy in AS2 is not equivalent to that of AS3. I ran
into this while porting the Mersenne Twister algorithm to AS2 - I
couldn't even store 2^32 as a hex value in AS2 (0x100000000, which
equals 4294967296).
At least we can be somewhat numerically accurate now...
good luck,
jon
On Sep 16, 2007, at 5:15 PM, Troy Gilbert wrote:
> Why does Math.cos(Math.PI/2) not return zero?
Round-off error in the Math libs? It does return a value very close to
0 (1.7xe-17).
Troy.
