Re: Dollarformat -vs- Numberformat
Thanks for the reply, I ended up finding the cluprit in a difference of SQL statements... one page had select round(line.price * line.rec_qty) and another had just sum(line.price * line.rec_qty)... the difference ended up being on the SQL site of things. Thanks again! Hatton Bud wrote: On 4/30/02, Hatton Humphrey penned: Here's the problem: I recently added a summary report to a financial calculation page. The page has been using NumberFormat(tot_tot+misc_tot,9-$999,999,999.99) for several years (the application has been in live use since around 1998). The report that I built is using DollarFormat instead. I was told that several of the totals are coming up fine in a few cases, off by a penny in most cases and off by two pennies in more than a few cases as well. First of all, am I correct in thinking that this is a display issue? Also, which one is more accurate? I think they pretty much work the same. The main thing you need to do is make sure you aren't calculating on a rounded number. You can't calculate on DollarFormat or NumberFormat with a dollar sign as it will return an error, so the problem may be in the tot_tot or misc_tot being calculated on DecimalFormat elsewhere. For instance. cfset tot_tot = 1.022 cfset num1 = tot_tot * 10 returns 10.22 correctly. cfset num2 = DecimalFormat(tot_tot) * 10 returns 10.20, which is off by 2 cents. __ Signup for the Fusion Authority news alert and keep up with the latest news in ColdFusion and related topics. http://www.fusionauthority.com/signup.cfm FAQ: http://www.thenetprofits.co.uk/coldfusion/faq Archives: http://www.mail-archive.com/cf-talk@houseoffusion.com/ Unsubscribe: http://www.houseoffusion.com/index.cfm?sidebar=lists
Dollarformat -vs- Numberformat
Okay, this may have been asked a thousand times but I'm going to make it 1001. When comparing the Numberformat and Decimalformat funcitons (for CF 4.x) what is the difference when it comes to rounding and numeric accuracy? Here's the problem: I recently added a summary report to a financial calculation page. The page has been using NumberFormat(tot_tot+misc_tot,9-$999,999,999.99) for several years (the application has been in live use since around 1998). The report that I built is using DollarFormat instead. I was told that several of the totals are coming up fine in a few cases, off by a penny in most cases and off by two pennies in more than a few cases as well. First of all, am I correct in thinking that this is a display issue? Also, which one is more accurate? Thanks! Hatton Humphrey __ Your ad could be here. Monies from ads go to support these lists and provide more resources for the community. http://www.fusionauthority.com/ads.cfm FAQ: http://www.thenetprofits.co.uk/coldfusion/faq Archives: http://www.mail-archive.com/cf-talk@houseoffusion.com/ Unsubscribe: http://www.houseoffusion.com/index.cfm?sidebar=lists
Re: Dollarformat -vs- Numberformat
On 4/30/02, Hatton Humphrey penned: Here's the problem: I recently added a summary report to a financial calculation page. The page has been using NumberFormat(tot_tot+misc_tot,9-$999,999,999.99) for several years (the application has been in live use since around 1998). The report that I built is using DollarFormat instead. I was told that several of the totals are coming up fine in a few cases, off by a penny in most cases and off by two pennies in more than a few cases as well. First of all, am I correct in thinking that this is a display issue? Also, which one is more accurate? I think they pretty much work the same. The main thing you need to do is make sure you aren't calculating on a rounded number. You can't calculate on DollarFormat or NumberFormat with a dollar sign as it will return an error, so the problem may be in the tot_tot or misc_tot being calculated on DecimalFormat elsewhere. For instance. cfset tot_tot = 1.022 cfset num1 = tot_tot * 10 returns 10.22 correctly. cfset num2 = DecimalFormat(tot_tot) * 10 returns 10.20, which is off by 2 cents. -- Bud Schneehagen - Tropical Web Creations _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ ColdFusion Solutions / eCommerce Development [EMAIL PROTECTED] http://www.twcreations.com/ 954.721.3452 __ Your ad could be here. Monies from ads go to support these lists and provide more resources for the community. http://www.fusionauthority.com/ads.cfm FAQ: http://www.thenetprofits.co.uk/coldfusion/faq Archives: http://www.mail-archive.com/cf-talk@houseoffusion.com/ Unsubscribe: http://www.houseoffusion.com/index.cfm?sidebar=lists
