Yuck. Okay, so is there any practical work-around for getting X.XX5 to
consistently round up? I suppose I could do something like add
0.0000001 to any number that I'm rounding, but that seems pretty sloppy.
Is there a best practice for dealing with this?
-William
Dossy Shiobara wrote:
On 2008.05.03, William Scott Jordan <[EMAIL PROTECTED]> wrote:
This is really more of a tcl question, but I'm hoping that someone on
the list might have an explanation. Why does [format %.2f 18.005] round
down to "18.00" and [format %.2f 1.415] round up to "1.42"? Any
guesses? Am I missing something obvious here?
Classic floating-point precision and rounding issue. See:
http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding
Read the third paragraph in that section.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]>
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
