Re: [ACFUG Discuss] Negative plus positive equals less than zero?
I may try that. My hackish solution was that if the resulting number was
less than 0 but greater than -0.01 to just return zero. I'm dealing with
money amounts here with no partial cents so I'm not really losing any
significant data. Decimalformat takes care of this but I need to have the
negative sign to the right of the number (client request) so I have to use
numberFormat instead, Thanks.
On 12/28/07, Cameron Childress [EMAIL PROTECTED] wrote:
I ran into this a number of times on a eCom system I was working on
recently. Gerry is right in that it's just a really small floating
point number. I've seen some financial systems deal with this problem
by shifting the decimal when dealing with numbers and shifting it back
only for display. In that case.
For example 62355.57 would be stored as 6235557 or even as 623555700
in financial systems that use 4 decimal points. You'd conduct all
math on these whole numbers and avoid floating point problems and then
just x/100 or x/1 to get it back to a decimal format (maybe in a
UDF) for display.
Another more hackish solution I've seen is to numberFormat() all
inputs to any math operation, which will strip off all the extra float
and give you a straighter answer. I'm not sure I would actually
suggest this cause it makes your code look like crap - but if you do
it the problem goes away. Try this out and see if your problem goes
away...
cfscript
total = 0;
numberList = 62355.57,-62355.57,-333.01,261.09 ,17.98,35.96,17.98;
for (i = 1;i lte listLen(numberList);i = i + 1)
{
writeOutput(#total# + #ListGetAt(numberList,i)# = );
total = numberFormat(total,9.99) +
numberFormat(ListGetAt(numberList,i),9.99);
writeOutput(#total#br);
}
writeOutput('numberFormat(total,999,999,999.99) = '
numberFormat(total,999,999,999.99));
/cfscript
-Cameron
On Dec 28, 2007 11:20 AM, Howard Fore [EMAIL PROTECTED] wrote:
Hey,
Can someone confirm this seemingly odd behavior? If I execute the
following
code:
cfscript
total = 0;
numberList = 62355.57,-62355.57,-333.01,261.09 ,17.98,35.96,17.98;
for (i = 1;i lte listLen(numberList);i = i + 1)
{
writeOutput(#total# + #ListGetAt(numberList,i)# = );
total = total + ListGetAt(numberList,i);
writeOutput(#total#br);
}
writeOutput('numberFormat(total,999,999,999.99) = '
numberFormat(total,999,999,999.99));
/cfscript
I get the following output:
0 + 62355.57 = 62355.57
62355.57 + -62355.57 = 0
0 + -333.01 = -333.01
-333.01 + 261.09 = -71.92
-71.92 + 17.98 = -53.94
-53.94 + 35.96 = -17.98
-17.98 + 17.98 = -1.06581410364E-014
numberFormat(total,999,999, 999.99) = -0.00
I've tried using javacasts to make sure that there wasn't some odd
string to
number conversion thing going on too and still got the same result. Any
ideas?
--
Howard Fore, [EMAIL PROTECTED]
Whether you believe you can do a thing or not, you are right. -- Henry
Ford
-
Annual Sponsor - Figleaf Software
To unsubscribe from this list, manage your profile @
http://www.acfug.org?fa=login.edituserform
For more info, see http://www.acfug.org/mailinglists
Archive @ http://www.mail-archive.com/discussion%40acfug.org/
List hosted by FusionLink
-
--
Cameron Childress
Sumo Consulting Inc
http://www.sumoc.com
---
cell: 678.637.5072
aim: cameroncf
email: [EMAIL PROTECTED]
-
Annual Sponsor FigLeaf Software - http://www.figleaf.com
To unsubscribe from this list, manage your profile @
http://www.acfug.org?fa=login.edituserform
For more info, see http://www.acfug.org/mailinglists
Archive @ http://www.mail-archive.com/discussion%40acfug.org/
List hosted by http://www.fusionlink.com
-
--
Howard Fore, [EMAIL PROTECTED]
Whether you believe you can do a thing or not, you are right. -- Henry
Ford
-
Annual Sponsor FigLeaf Software - http://www.figleaf.com
To unsubscribe from this list, manage your profile @
http://www.acfug.org?fa=login.edituserform
For more info, see http://www.acfug.org/mailinglists
Archive @ http://www.mail-archive.com/discussion%40acfug.org/
List hosted by http://www.fusionlink.com
-
RE: [ACFUG Discuss] Negative plus positive equals less than zero?
I think I would write a UDF that would round all of your floating point
calculations. Don't forget that you have to worry about +- 0.001
as well as +- 0.001
Try playing around with this
cfset val=-0.91
cfset decimal=3
Make this part a UDF with val and decimal as arguments.
cfset multiplier=10^decimal
cfset newval=round(multiplier*val)/multiplier
Output or return this new value
cfoutput#newval#/cfoutput
Gerry Gurevich
Application Development
NIEHS ITSS Contractor
Lockheed Martin Information Technology
919-361-5444 ext 311
From: Howard Fore [mailto:[EMAIL PROTECTED]
Sent: Friday, December 28, 2007 12:55 PM
To: discussion@acfug.org
Subject: Re: [ACFUG Discuss] Negative plus positive equals less than
zero?
I may try that. My hackish solution was that if the resulting number was
less than 0 but greater than -0.01 to just return zero. I'm dealing with
money amounts here with no partial cents so I'm not really losing any
significant data. Decimalformat takes care of this but I need to have
the negative sign to the right of the number (client request) so I have
to use numberFormat instead, Thanks.
On 12/28/07, Cameron Childress [EMAIL PROTECTED] wrote:
I ran into this a number of times on a eCom system I was working on
recently. Gerry is right in that it's just a really small floating
point number. I've seen some financial systems deal with this problem
by shifting the decimal when dealing with numbers and shifting it back
only for display. In that case.
For example 62355.57 would be stored as 6235557 or even as 623555700
in financial systems that use 4 decimal points. You'd conduct all
math on these whole numbers and avoid floating point problems and then
just x/100 or x/1 to get it back to a decimal format (maybe in a
UDF) for display.
Another more hackish solution I've seen is to numberFormat() all
inputs to any math operation, which will strip off all the extra float
and give you a straighter answer. I'm not sure I would actually
suggest this cause it makes your code look like crap - but if you do
it the problem goes away. Try this out and see if your problem goes
away...
cfscript
total = 0;
numberList = 62355.57,-62355.57,-333.01,261.09 ,17.98,35.96,17.98;
for (i = 1;i lte listLen(numberList);i = i + 1)
{
writeOutput(#total# + #ListGetAt(numberList,i)# = );
total = numberFormat(total,9.99) +
numberFormat(ListGetAt(numberList,i),9.99);
writeOutput(#total#br);
}
writeOutput('numberFormat(total,999,999,999.99) = '
numberFormat(total,999,999,999.99));
/cfscript
-Cameron
On Dec 28, 2007 11:20 AM, Howard Fore [EMAIL PROTECTED] wrote:
Hey,
Can someone confirm this seemingly odd behavior? If I execute the
following
code:
cfscript
total = 0;
numberList = 62355.57,-62355.57,-333.01,261.09 ,17.98,35.96,17.98;
for (i = 1;i lte listLen(numberList);i = i + 1)
{
writeOutput(#total# + #ListGetAt(numberList,i)# = );
total = total + ListGetAt(numberList,i);
writeOutput(#total#br);
}
writeOutput('numberFormat(total,999,999,999.99) = '
numberFormat(total,999,999, 999.99));
/cfscript
I get the following output:
0 + 62355.57 = 62355.57
62355.57 + -62355.57 = 0
0 + -333.01 = -333.01
-333.01 + 261.09 = -71.92
-71.92 + 17.98 = -53.94
-53.94 + 35.96 = -17.98
-17.98 + 17.98 = -1.06581410364E-014
numberFormat(total,999,999, 999.99) = -0.00
I've tried using javacasts to make sure that there wasn't some odd
string to
number conversion thing going on too and still got the same result.
Any
ideas?
--
Howard Fore, [EMAIL PROTECTED]
Whether you believe you can do a thing or not, you are right. --
Henry
Ford
-
Annual Sponsor - Figleaf Software
To unsubscribe from this list, manage your profile @
http://www.acfug.org?fa=login.edituserform
For more info, see http://www.acfug.org/mailinglists
Archive @ http://www.mail-archive.com/discussion%40acfug.org/
List hosted by FusionLink
-
--
Cameron Childress
Sumo Consulting Inc
http://www.sumoc.com
---
cell: 678.637.5072
aim: cameroncf
email: [EMAIL PROTECTED]
-
Annual Sponsor FigLeaf Software - http://www.figleaf.com
To unsubscribe from this list, manage your profile @
http://www.acfug.org?fa=login.edituserform
For more info, see http://www.acfug.org/mailinglists
Archive @ http://www.mail-archive.com/discussion%40acfug.org/
List hosted by http://www.fusionlink.com
-
--
Howard Fore, [EMAIL PROTECTED]
Whether you believe you can do a thing or not, you are right. -- Henry
Ford
-
Annual Sponsor - Figleaf Software http://www.figleaf.com
