I ran into this a number of times on a eCom system I was working on
recently. Gerry is right in that it's just a really small floating
point number. I've seen some financial systems deal with this problem
by shifting the decimal when dealing with numbers and shifting it back
only for display. In that case.
For example 62355.57 would be stored as 6235557 or even as 623555700
in financial systems that use 4 decimal points. You'd conduct all
math on these whole numbers and avoid floating point problems and then
just x/100 or x/10000 to get it back to a decimal format (maybe in a
UDF) for display.
Another more hackish solution I've seen is to numberFormat() all
inputs to any math operation, which will strip off all the extra float
and give you a straighter answer. I'm not sure I would actually
suggest this cause it makes your code look like crap - but if you do
it the problem goes away. Try this out and see if your problem goes
away...
<cfscript>
total = 0;
numberList = "62355.57,-62355.57,-333.01,261.09 ,17.98,35.96,17.98";
for (i = 1;i lte listLen(numberList);i = i + 1)
{
writeOutput("#total# + #ListGetAt(numberList,i)# = ");
total = numberFormat(total,"999999999.99") +
numberFormat(ListGetAt(numberList,i),"999999999.99");
writeOutput("#total#<br>");
}
writeOutput('numberFormat(total,"999,999,999.99") = ' &
numberFormat(total,"999,999,999.99"));
</cfscript>
-Cameron
On Dec 28, 2007 11:20 AM, Howard Fore <[EMAIL PROTECTED]> wrote:
> Hey,
>
> Can someone confirm this seemingly odd behavior? If I execute the following
> code:
>
> <cfscript>
>
> total = 0;
> numberList = "62355.57,-62355.57,-333.01,261.09 ,17.98,35.96,17.98";
> for (i = 1;i lte listLen(numberList);i = i + 1)
> {
> writeOutput("#total# + #ListGetAt(numberList,i)# = ");
> total = total + ListGetAt(numberList,i);
> writeOutput("#total#<br>");
>
> }
> writeOutput('numberFormat(total,"999,999,999.99") = ' &
> numberFormat(total,"999,999,999.99"));
> </cfscript>
>
> I get the following output:
>
> 0 + 62355.57 = 62355.57
> 62355.57 + -62355.57 = 0
> 0 + -333.01 = -333.01
> -333.01 + 261.09 = -71.92
> -71.92 + 17.98 = -53.94
> -53.94 + 35.96 = -17.98
> -17.98 + 17.98 = -1.06581410364E-014
> numberFormat(total,"999,999, 999.99") = -0.00
>
> I've tried using javacasts to make sure that there wasn't some odd string to
> number conversion thing going on too and still got the same result. Any
> ideas?
>
> --
> Howard Fore, [EMAIL PROTECTED]
> "Whether you believe you can do a thing or not, you are right." -- Henry
> Ford
> -------------------------------------------------------------
> Annual Sponsor - Figleaf Software
>
> To unsubscribe from this list, manage your profile @
> http://www.acfug.org?fa=login.edituserform
>
> For more info, see http://www.acfug.org/mailinglists
> Archive @ http://www.mail-archive.com/discussion%40acfug.org/
> List hosted by FusionLink
> -------------------------------------------------------------
--
Cameron Childress
Sumo Consulting Inc
http://www.sumoc.com
---
cell: 678.637.5072
aim: cameroncf
email: [EMAIL PROTECTED]
-------------------------------------------------------------
Annual Sponsor FigLeaf Software - http://www.figleaf.com
To unsubscribe from this list, manage your profile @
http://www.acfug.org?fa=login.edituserform
For more info, see http://www.acfug.org/mailinglists
Archive @ http://www.mail-archive.com/discussion%40acfug.org/
List hosted by http://www.fusionlink.com
-------------------------------------------------------------
