That's the explanation I've been looking for. I think we were really close to it a few times and just missed it (for example the "not enough precision" comment). So this isn't a bug as much as a limitation of the definition of Number (which I think is based on a standard implementation of float?).
The original problem should have been that i was getting: Math.abs(Number.MAX_VALUE - 1290) == Math.abs(Number.MAX_VALUE - 1284); // returns true I wounder if I only imagined Math.abs() making a difference (I thought I tested the difference)? Either way, int.MAX_VALUE has been working so I'm happy. Thanks, - Daniel Freiman On Fri, Jun 13, 2008 at 3:43 PM, Gordon Smith <[EMAIL PROTECTED]> wrote: > Now that you're using Number.MAX_VALUE rather than int.MAX_VALUE, this > difference *should* evaluate to 0. > > > > The reason is that Number.MAX_VALUE is a huge value, approximately > 1.79769313486231e+308, with 309 digits to the left of the decimal point. > Number, being only 64 bits, can't represent all of these... it represents > the 15 or 16 most-significant digits, with an exponent telling where the > decimal point goes. The next-to-the-maximum value that Number can represent > is something like 1.79769313486230e+308, where the least-significant of the > digits stores is a 0 instead of a 1. (Actually they're stored in binary, not > decimal, but that's another story.) > > > > So when you subtract a puny value like 1290, it's like doing > > > > 179769313486231??????????????????????????????????????????????...???? > > > -1290 > > > --------------------------------------------------------------------------------------------------------------------- > > 179769313486231??????????????????????????????????????????????...???? > > > > You get the same thing you started with, because the mathematically exact > result is closer to the maximum value than to the next-to-the-maximum value > of Number. If you subtract a much larger value like 1e+294 you'll start > affecting the least significant digit. > > > > Gordon Smith > > Adobe Flex SDK Team > > > ------------------------------ > > *From:* flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] *On > Behalf Of *Daniel Freiman > *Sent:* Friday, June 13, 2008 12:26 PM > *To:* flexcoders@yahoogroups.com > *Subject:* Re: [flexcoders] Re: Math.abs() Limitation? > > > > I'll have to double check, but I'm don't think the code was doing the right > thing either, because: > > Math.abs(Number.MAX_VALUE - 1290) - Math.abs(Number.MAX_VALUE - 1284); // > returns 0 > > > On Fri, Jun 13, 2008 at 1:58 PM, Mike Morearty <[EMAIL PROTECTED]> > wrote: > > Daniel, this is a bug in Flex Builder, which occurs when displaying the > values of Numbers > whose value is greater than 0x7FFFFFFFFFFFFFFF. I'll get a fix in ASAP. > Thanks for the > feedback, and sorry for the inconvenience. > > To be clear, the Flex Builder debugger's Variables view is showing the > wrong value, but the > code is doing the right thing, and trace() is showing the right thing. > > - Mike Morearty, Adobe Flex Builder team > > > > --- In flexcoders@yahoogroups.com <flexcoders%40yahoogroups.com>, "Daniel > Freiman" <[EMAIL PROTECTED]> wrote: > > > > What did I mean? I'm not sure. For some reason I thought that floats > loose > > precision at near the end of their range, but even if that is true, that > > wouldn't explain the lose of precision that large. In any event, I > retried > > the following code: > > > > var t1:Number = Number.MAX_VALUE; > > var t3:Number = Number.MAX_VALUE - 1290; > > var t4:Number = Number.MAX_VALUE - 1284; > > var t5:Number = Math.abs(Number.MAX_VALUE - 1290); > > var t6:Number = Math.abs(Number.MAX_VALUE - 1284); > > > > var a1:Number = t3 - t4; > > var a2:Number = Math.abs(t3 - t4); > > var a3:Number = t5 - t6; > > var a4:Number = (Number.MAX_VALUE - 1290) - (Number.MAX_VALUE - > > 1284); > > var a5:Number = Math.abs(Number.MAX_VALUE - 1290) - > > Math.abs(Number.MAX_VALUE - 1284); > > var a6:Number = Math.abs((Number.MAX_VALUE - 1290) - > > (Number.MAX_VALUE - 1284)); > > > > > > The debugger reported all of the first group of variable values as > > "9223372036854775807 [0x7fffffffffffffff]" and the traces of these values > > reported all of the values as "1.79769313486231e+308" > > This is not the same results that I remember getting previously where t3 > != > > t4. The second group all have values of 0. When i have time I'll look > into > > this more. > > > > - Daniel Freiman > > > > > On Wed, Jun 11, 2008 at 2:04 AM, Gordon Smith <[EMAIL PROTECTED]> wrote: > > > > > > Number.MAX_VALUE doesn't have enough precision to handle what I was > > > trying to do. > > > > > > What do you mean by this? A Number, being 64 bits, actually has both > more > > > precision and more range than int, which is 32 bits. For example, in > > > addition to storing fractional values, it can store integers much > larger > > > than int.MAX_VALUE. > > > > > > > > > > > > Gordon Smith > > > > > > Adobe Flex SDK Team > > > > > > > > > ------------------------------ > > > > > > *From:* flexcoders@yahoogroups.com <flexcoders%40yahoogroups.com>[mailto: > flexcoders@yahoogroups.com <flexcoders%40yahoogroups.com>] *On > > > Behalf Of *Daniel Freiman > > > *Sent:* Friday, June 06, 2008 2:10 PM > > > *To:* flexcoders@yahoogroups.com <flexcoders%40yahoogroups.com> > > > *Subject:* Re: [flexcoders] Math.abs() Limitation? > > > > > > > > > > > > I figured it out. The trace made it obvious (as opposed the debugger > which > > > is what I was previously using). I was using Number.MAX_VALUE, not > > > int.MAX_VALUE. Number.MAX_VALUE doesn't have enough precision to handle > > > what I was trying to do. > > > > > > Thanks, > > > > > > - Daniel Freiman > > > > > > > On Fri, Jun 6, 2008 at 2:29 PM, Gordon Smith <[EMAIL PROTECTED]> wrote: > > > > > > What do you get when you trace out the following values? > > > > > > > > > > > > int.MAX_VALUE - 1290 > > > > > > Math.abs(int.MAX_VALUE - 1290) > > > > > > int.MAX_VALUE - 1284 > > > > > > Math.abs(int.MAX_VALUE - 1284) > > > > > > > > > > > > Gordon Smith > > > > > > Adobe Flex SDK Team > > > > > > > > > ------------------------------ > > > > > > *From:* flexcoders@yahoogroups.com <flexcoders%40yahoogroups.com>[mailto: > flexcoders@yahoogroups.com <flexcoders%40yahoogroups.com>] *On > > > Behalf Of *Alex Harui > > > *Sent:* Friday, June 06, 2008 11:18 AM > > > *To:* flexcoders@yahoogroups.com <flexcoders%40yahoogroups.com> > > > *Subject:* RE: [flexcoders] Math.abs() Limitation? > > > > > > > > > > > > What if you use temporary variables? > > > > > > > > > ------------------------------ > > > > > > *From:* flexcoders@yahoogroups.com <flexcoders%40yahoogroups.com>[mailto: > flexcoders@yahoogroups.com <flexcoders%40yahoogroups.com>] *On > > > Behalf Of *Daniel Freiman > > > *Sent:* Friday, June 06, 2008 9:09 AM > > > *To:* flexcoders > > > *Subject:* [flexcoders] Math.abs() Limitation? > > > > > > > > > > > > According to my code the following statement returns true. > > > > > > Math.abs(int.MAX_VALUE - 1290) == Math.abs(int.MAX_VALUE - 1284) > > > > > > I tried converting everything to type Number that didn't help. Getting > > > ride of the abs() makes the calculation work correctly but then I don't > have > > > the absolute value. > > > > > > Thoughts? > > > > > > - Daniel Freiman > > > > > > > > > > > > > > > > > > > > > >
