to prevent anyone answering on this thread:
after toying with it again, and reading the docs + wikipedia I figured
out that I can only trust a result of 15 digits or less, unless I have
the string be returned instead of the number.
n 12/18/2010 8:34 PM, Anthony Pace wrote:
function floatSumV
whoa... hold on Help Please.
After I posted yesterday, (or the night before??? can't remember) as I
realized how badly the code sucked, I was just messing around with this
code below to see if I could separate the halves and recompose them
properly; however,
if you pop the code below on t
Oh yeah, I forgot to give some examples to test it with, so...
trace(floatSum(-0.1,0.9155,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1));
//vs
trace(-0.1+0.9155+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1);
//and
trace(floatProduct(2.1,0.1,1.0001,.0001));
//vs
trace(2.1*0.1*1.0001*.0001);
On 12/15/2010 2:25 PM, Anthony
I wrote the functions below in literally just a few min, so, even though
they do seem to work for me, I wouldn't necessarily say they are
production ready.
Also, I would only use these if you will know that the numbers past the
mantissa are of small amounts.
Oh yeah, and please tell me if I
Adrian Zając wrote:
>trace (0.27 + 0.03); // output --> 0.30004
>
> Can anyone tell me why I get this weird result in output window?
As people have said, it's a problem with decimals. It's not a problem with
Flash--it's a problem with binary numbers.
Integers are accurate bec
Thanks Zeh, now I understand.
So do we have to keep an eye on our variables so we don't have for
example:7.998 * 0,3004 ?
Because I think it is a little heavier for processors to count than: 8
* 0.3
W dniu 2010-12-14 16:36, Zeh Fernando pisze:
I like to quote t
tom rhodes skriver:
yup interesting that the old AVM gives you what you'd think...
Pure luck, you might have compiled against something that happened to
drop the decimal instead .
___
Flashcoders mailing list
Flashcoders@chattyfig.figleaf.com
http:
yup interesting that the old AVM gives you what you'd think...
On 14 December 2010 16:36, Zeh Fernando wrote:
> I like to quote this, from PHP.net's "Floating Point" documentation:
>
> "Floating point numbers have limited precision. Although it depends on the
> system, PHP typically uses the IE
Howdy Adrian,
Here is one way.
trace (int((0.27 + 0.03)*100)/100) // output --> 0.3
Later,
John
on 12/14/10 8:42 AM, Adrian Zając at zajac.adr...@gmail.com wrote:
> Hello,
>
> First of all, I want to say Hi to everyone here. This is my first post.
>
>
> Please, take a look at this part
I like to quote this, from PHP.net's "Floating Point" documentation:
"Floating point numbers have limited precision. Although it depends on the
system, PHP typically uses the IEEE 754 double precision format, which will
give a maximum relative error due to rounding in the order of 1.11e-16. Non
el
It's an issue with floating point accuracy/calculations,
Here is some info:
http://joshblog.net/2007/01/30/flash-floating-point-number-errors/
saw more about it somewhere else... don't think it's an issue with the FP tho'.
- Karim
On 14 Dec 2010, at 15:15, tom rhodes wrote:
> same here compil
If you need the addition to be accurate and you know that the numbers will
always be decimals you can multiply the numbers by 100 and then you will add 27
+ 3 which will return 30 and then divide it by 100 to get your .3 (or at least
I think that will work as I have not tried it)
Yes St
same here compiling for flash player 10 and flash player 9, 8 and below give
0.3 as expected
On 14 December 2010 15:42, Adrian Zając wrote:
> trace (0.27 + 0.03);
___
Flashcoders mailing list
Flashcoders@chattyfig.figleaf.com
http://chattyfig.figl
-boun...@chattyfig.figleaf.com] On Behalf Of Adrian Zajac
Sent: Tuesday, December 14, 2010 8:43 AM
To: Flash Coders List
Subject: [Flashcoders] problem with adding two digits
Hello,
First of all, I want to say Hi to everyone here. This is my first post.
Please, take a look at this part of code
Hello,
First of all, I want to say Hi to everyone here. This is my first post.
Please, take a look at this part of code:
trace (0.27 + 0.03); // output --> 0.30004
Can anyone tell me why I get this weird result in output window?
Regards
Adrian
_
15 matches
Mail list logo
