Re: [flexcoders] ActionsScript 3 Cookbook Companion Classes: Number Utilities Bug
This is a floating point bug. See Gordon's response to Is this a bug? About Number type on Aug 30 for full details. I'll quote the important parts: getting 23.703 isn't a bug; you'd get the same answer in other languages like Java and C++. The imprecision is because Number in ActionScript (like float in Java and C++) stores a floating-point value as a binary fraction, not as a decimal fraction. These datatypes use binary fractions because that's how current microprocessors store and manipulate floating-point values...Perhaps you can simply round your results to the precision you need? Take a look at the toFixed() method of the Number class. - Dan On 12/6/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Wow, the description says that this number will be rounded up to the next decimal point: trace (NumberUtilities.round(3.75,.1)); .1, being the flag to round up, so the result should be 3.8. It's not it's: 3.8003! I know I'm using this method correctly, any tips or suggestions. Thanks, Patrick
RE: [flexcoders] ActionsScript 3 Cookbook Companion Classes: Number Utilities Bug
The Number type can't store the value 3.8 exactly, because as a binary fraction it would require an infinite number of bits. You're getting the Number closest to 3.8. - Gordon From: flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Wednesday, December 06, 2006 2:12 PM To: flexcoders@yahoogroups.com Subject: [flexcoders] ActionsScript 3 Cookbook Companion Classes: Number Utilities Bug Wow, the description says that this number will be rounded up to the next decimal point: trace (NumberUtilities.round(3.75,.1)); .1, being the flag to round up, so the result should be 3.8. It's not it's: 3.8003! I know I'm using this method correctly, any tips or suggestions. Thanks, Patrick
