What did I mean? I'm not sure. For some reason I thought that floats loose
precision at near the end of their range, but even if that is true, that
wouldn't explain the lose of precision that large. In any event, I retried
the following code:
var t1:Number = Number.MAX_VALUE;
var t3:Number = Number.MAX_VALUE - 1290;
var t4:Number = Number.MAX_VALUE - 1284;
var t5:Number = Math.abs(Number.MAX_VALUE - 1290);
var t6:Number = Math.abs(Number.MAX_VALUE - 1284);
var a1:Number = t3 - t4;
var a2:Number = Math.abs(t3 - t4);
var a3:Number = t5 - t6;
var a4:Number = (Number.MAX_VALUE - 1290) - (Number.MAX_VALUE -
1284);
var a5:Number = Math.abs(Number.MAX_VALUE - 1290) -
Math.abs(Number.MAX_VALUE - 1284);
var a6:Number = Math.abs((Number.MAX_VALUE - 1290) -
(Number.MAX_VALUE - 1284));
The debugger reported all of the first group of variable values as
"9223372036854775807 [0x7fffffffffffffff]" and the traces of these values
reported all of the values as "1.79769313486231e+308"
This is not the same results that I remember getting previously where t3 !=
t4. The second group all have values of 0. When i have time I'll look into
this more.
- Daniel Freiman
On Wed, Jun 11, 2008 at 2:04 AM, Gordon Smith <[EMAIL PROTECTED]> wrote:
> > Number.MAX_VALUE doesn't have enough precision to handle what I was
> trying to do.
>
> What do you mean by this? A Number, being 64 bits, actually has both more
> precision and more range than int, which is 32 bits. For example, in
> addition to storing fractional values, it can store integers much larger
> than int.MAX_VALUE.
>
>
>
> Gordon Smith
>
> Adobe Flex SDK Team
>
>
> ------------------------------
>
> *From:* flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] *On
> Behalf Of *Daniel Freiman
> *Sent:* Friday, June 06, 2008 2:10 PM
> *To:* flexcoders@yahoogroups.com
> *Subject:* Re: [flexcoders] Math.abs() Limitation?
>
>
>
> I figured it out. The trace made it obvious (as opposed the debugger which
> is what I was previously using). I was using Number.MAX_VALUE, not
> int.MAX_VALUE. Number.MAX_VALUE doesn't have enough precision to handle
> what I was trying to do.
>
> Thanks,
>
> - Daniel Freiman
>
> On Fri, Jun 6, 2008 at 2:29 PM, Gordon Smith <[EMAIL PROTECTED]> wrote:
>
> What do you get when you trace out the following values?
>
>
>
> int.MAX_VALUE - 1290
>
> Math.abs(int.MAX_VALUE - 1290)
>
> int.MAX_VALUE - 1284
>
> Math.abs(int.MAX_VALUE - 1284)
>
>
>
> Gordon Smith
>
> Adobe Flex SDK Team
>
>
> ------------------------------
>
> *From:* flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] *On
> Behalf Of *Alex Harui
> *Sent:* Friday, June 06, 2008 11:18 AM
> *To:* flexcoders@yahoogroups.com
> *Subject:* RE: [flexcoders] Math.abs() Limitation?
>
>
>
> What if you use temporary variables?
>
>
> ------------------------------
>
> *From:* flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] *On
> Behalf Of *Daniel Freiman
> *Sent:* Friday, June 06, 2008 9:09 AM
> *To:* flexcoders
> *Subject:* [flexcoders] Math.abs() Limitation?
>
>
>
> According to my code the following statement returns true.
>
> Math.abs(int.MAX_VALUE - 1290) == Math.abs(int.MAX_VALUE - 1284)
>
> I tried converting everything to type Number that didn't help. Getting
> ride of the abs() makes the calculation work correctly but then I don't have
> the absolute value.
>
> Thoughts?
>
> - Daniel Freiman
>
>
>
>
>
