Hi Michael, In floating point calculation, 1.0-.9 is not exactly 0.1. This is easily seen by subtracting. > (1.0-.9)-0.1 [1] -2.775558e-17 > (1.0-.9)==0.1 [1] FALSE
David is right, you can't "correct" this. You can only compensate by taking care that you never, ever test whether 2 FP numbers are equal, because they almost never are. You must always ask whether the difference is small. > round(1.0-.9-.1,15)==0 [1] TRUE Unfortunately, most of us forget this rule once in a while and write a loop like "while (x!=0)..." that won't terminate. HTH Rex -----Original Message----- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Folkes, Michael Sent: Thursday, March 03, 2011 9:24 PM To: r-help@r-project.org Subject: [R] Floating points and floor() ? Perhaps somebody could clarify for me if the following is a floating point matter or otherwise, and how am I to correct for it? > floor(100*.1) [1] 10 > 100*(1.0-.9) [1] 10 > floor(100*(1-0.9)) [1] 9 Thanks! Michael _______________________________________________________ Michael Folkes Salmon Stock Assessment Canadian Dept. of Fisheries & Oceans Pacific Biological Station 3190 Hammond Bay Rd. Nanaimo, B.C., Canada V9T-6N7 Ph (250) 756-7264 Fax (250) 756-7053 michael.fol...@dfo-mpo.gc.ca [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. message may contain confidential information. If you are not the designated recipient, please notify the sender immediately, and delete the original and any copies. Any use of the message by you is prohibited. ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
