Re: [Flashcoders] floating point variables...
It was a fluke... it broke easily with addition; yet, multiplication seems to work perfectly and division rounds up the last variable. Weird that things worked a dozen times, and then just stopped. Any clues? Anthony Pace wrote: Am I losing it or has Adobe finally actually somewhat fixed the floating point inaccuracy? I remember them discussing it at FITC, but I thought it was going to be in the next release to keep up with ECMA. I am testing it out now, and things seem to be working great; is this just a fluke on my system, or does this work for everyone? Thank You, Anthony Pace ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Floating point pain
On 6/5/07, Merrill, Jason <[EMAIL PROTECTED]> wrote: Thanks, yeah, I thought of that. I was hoping there was some unknown Math function I had missed. The only other one I can think of is modulo. It would be slightly more readable (IMHO) and fit into one line, but would still require the Math.round, * 100, / 100 thing due to the imprecision of floats. var fract = Math.round(( val % 1 ) * 100) / 100; Mark ___ Flashcoders@chattyfig.figleaf.com To change your subscription options or search the archive: http://chattyfig.figleaf.com/mailman/listinfo/flashcoders Brought to you by Fig Leaf Software Premier Authorized Adobe Consulting and Training http://www.figleaf.com http://training.figleaf.com
RE: [Flashcoders] Floating point pain
Thanks, yeah, I thought of that. I was hoping there was some unknown Math function I had missed. Jason Merrill Bank of America GT&O Learning & Leadership Development eTools & Multimedia Team >>-Original Message- >>From: [EMAIL PROTECTED] >>[mailto:[EMAIL PROTECTED] On Behalf >>Of Glen Pike >>Sent: Tuesday, June 05, 2007 4:18 PM >>To: flashcoders@chattyfig.figleaf.com >>Subject: Re: [Flashcoders] Floating point pain >> >>Parse it as a string then turn it into a number again :) >> >> >> >>Merrill, Jason wrote: >>> If I have a floating point value, say, 25.81 and want to >>separate the >>> first two places beyond the decimal point (.81) from 25.81, >>I thought >>> simple subtration would work, but it seems floating points are even >>> less accurate in Flash than I thought. I know the are >>imprecise, but >>> are they really this bad? >>> >>> so: >>> >>> var a = 25.81 >>> var b = Math.floor(a) >>> var c = a-b >>> var d = (Math.round((a-b)*100))*.01 >>> trace(a) //25.81 as expected >>> trace(b) //25 as expected >>> trace(c) //returns 0.80999... >>> trace(d) //returns 0.81 - what I want and expect >>> >>> Is my last method for var "d" the most pratical, precise, >>and readable >>> way to get 0.81 from 25.81? >>> >>> Jason Merrill >>> Bank of America >>> GT&O Learning & Leadership Development eTools & Multimedia Team >>> >>> >>> >>> ___ >>> Flashcoders@chattyfig.figleaf.com >>> To change your subscription options or search the archive: >>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >>> >>> Brought to you by Fig Leaf Software >>> Premier Authorized Adobe Consulting and Training >>> http://www.figleaf.com http://training.figleaf.com >>> >>> >>> >>___ >>Flashcoders@chattyfig.figleaf.com >>To change your subscription options or search the archive: >>http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >> >>Brought to you by Fig Leaf Software >>Premier Authorized Adobe Consulting and Training >>http://www.figleaf.com http://training.figleaf.com >> ___ Flashcoders@chattyfig.figleaf.com To change your subscription options or search the archive: http://chattyfig.figleaf.com/mailman/listinfo/flashcoders Brought to you by Fig Leaf Software Premier Authorized Adobe Consulting and Training http://www.figleaf.com http://training.figleaf.com
Re: [Flashcoders] Floating point pain
Parse it as a string then turn it into a number again :) Merrill, Jason wrote: If I have a floating point value, say, 25.81 and want to separate the first two places beyond the decimal point (.81) from 25.81, I thought simple subtration would work, but it seems floating points are even less accurate in Flash than I thought. I know the are imprecise, but are they really this bad? so: var a = 25.81 var b = Math.floor(a) var c = a-b var d = (Math.round((a-b)*100))*.01 trace(a) //25.81 as expected trace(b) //25 as expected trace(c) //returns 0.80999... trace(d) //returns 0.81 - what I want and expect Is my last method for var "d" the most pratical, precise, and readable way to get 0.81 from 25.81? Jason Merrill Bank of America GT&O Learning & Leadership Development eTools & Multimedia Team ___ Flashcoders@chattyfig.figleaf.com To change your subscription options or search the archive: http://chattyfig.figleaf.com/mailman/listinfo/flashcoders Brought to you by Fig Leaf Software Premier Authorized Adobe Consulting and Training http://www.figleaf.com http://training.figleaf.com ___ Flashcoders@chattyfig.figleaf.com To change your subscription options or search the archive: http://chattyfig.figleaf.com/mailman/listinfo/flashcoders Brought to you by Fig Leaf Software Premier Authorized Adobe Consulting and Training http://www.figleaf.com http://training.figleaf.com
Re: [Flashcoders] floating point
k, thanks all.
i went for the boundary check.
the other one was ""+i == "0.8" which was even dirtier ;)
At 12:33 PM 12/7/2005, Danny Kodicek wrote:
- Original Message - From: "Toon Van de Putte" <[EMAIL PROTECTED]>
To: "Flashcoders mailing list"
Sent: Monday, December 05, 2005 10:10 PM
Subject: Re: [Flashcoders] floating point
var i = 0;
var j= 0.8;
j*=10
onEnterFrame = function () {
i = i + 1;
trace (i);
if (i == j) trace("here");
}
Or is that just not possible (ie you don't know how many decimal spaces j
will have)?
The other way is something like
if (Math.abs(i-j)<0.001) {}
Danny
___
Flashcoders mailing list
Flashcoders@chattyfig.figleaf.com
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
___
Flashcoders mailing list
Flashcoders@chattyfig.figleaf.com
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] floating point
This has to do with the fact
that not every decimal value can be exactly
represented by a binary floating point. So the results
of adding a bunch of binary numbers that is not exact
to begin with will come out to be
different than the actual binary floating point for 0.8.
You can either turn the comparison into an integral
one as some has proposed or introduce
an absolute error (epsilon) into the comparison:
if (Math.abs(i - 0.8) < 0.1) trace("here");
- boon
Hans Wichman <[EMAIL PROTECTED]> wrote: Hi peeps,
related discussions have been on here before, but i didn't find a real answer.
Imagine this piece of code:
var i = 0;
onEnterFrame = function () {
i = i + 0.1;
trace (i);
if (i == 0.7) trace("here");
}
as you might expect, after running for 7 frames, it traces "here".
However if you change 0.7 to 0.8, it stops working.
I assume this is to floating point math and rounding errors and stuff?
In what way can i check for equality to 0.7 then? With a bigger than,
smaller than construction?
This seems so weird to me...
greetz
Hans
___
Flashcoders mailing list
Flashcoders@chattyfig.figleaf.com
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
-
Yahoo! Personals
Let fate take it's course directly to your email.
See who's waiting for you Yahoo! Personals
___
Flashcoders mailing list
Flashcoders@chattyfig.figleaf.com
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] floating point
- Original Message -
From: "Toon Van de Putte" <[EMAIL PROTECTED]>
To: "Flashcoders mailing list"
Sent: Monday, December 05, 2005 10:10 PM
Subject: Re: [Flashcoders] floating point
var i = 0;
var j= 0.8;
j*=10
onEnterFrame = function () {
i = i + 1;
trace (i);
if (i == j) trace("here");
}
Or is that just not possible (ie you don't know how many decimal spaces j
will have)?
The other way is something like
if (Math.abs(i-j)<0.001) {}
Danny
___
Flashcoders mailing list
Flashcoders@chattyfig.figleaf.com
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] floating point
var i = 0;
var j= 0.8;
j*=10
onEnterFrame = function () {
i = i + 1;
trace (i);
if (i == j) trace("here");
}
Or is that just not possible (ie you don't know how many decimal spaces j
will have)?
On 12/4/05, Hans Wichman <[EMAIL PROTECTED]> wrote:
>
> Hi peeps,
> related discussions have been on here before, but i didn't find a real
> answer.
> Imagine this piece of code:
> var i = 0;
>
> onEnterFrame = function () {
> i = i + 0.1;
> trace (i);
> if (i == 0.7) trace("here");
> }
>
> as you might expect, after running for 7 frames, it traces "here".
> However if you change 0.7 to 0.8, it stops working.
>
> I assume this is to floating point math and rounding errors and stuff?
> In what way can i check for equality to 0.7 then? With a bigger than,
> smaller than construction?
>
> This seems so weird to me...
>
> greetz
> Hans
>
> ___
> Flashcoders mailing list
> Flashcoders@chattyfig.figleaf.com
> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
>
--
Toon Van de Putte
___
Flashcoders mailing list
Flashcoders@chattyfig.figleaf.com
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] floating point
On 12/4/05, Hans Wichman <[EMAIL PROTECTED]> wrote:
>
> Hi peeps,
> related discussions have been on here before, but i didn't find a real
> answer.
> Imagine this piece of code:
> var i = 0;
>
> onEnterFrame = function () {
> i = i + 0.1;
> trace (i);
> if (i == 0.7) trace("here");
> }
>
> as you might expect, after running for 7 frames, it traces "here".
> However if you change 0.7 to 0.8, it stops working.
>
> I assume this is to floating point math and rounding errors and stuff?
> In what way can i check for equality to 0.7 then? With a bigger than,
> smaller than construction?
>
> This seems so weird to me...
>
This will check to make sure the numbers are equal to within 2 decimal
places. change both 100's to set how many decimal places you want to check.
if(Math.round(i*100) == (0.8 *100))
-David R
___
Flashcoders mailing list
Flashcoders@chattyfig.figleaf.com
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
