One more thing: this seems to happen because realvalue is 9.9499999999999993, and rounder is 0.050000000000000003, and adding them together produces exactly 10.000000000000000! Other platforms or libraries may be producing slightly different values which ended up being a hair less than 10 or a hair more. An unlucky value indeed! You flipped a quarter, and it landed on the edge.
--Ned. http://nedbatchelder.com -----Original Message----- From: Ned Batchelder [mailto:[EMAIL PROTECTED] Sent: Tuesday, 30 August, 2005 3:04 PM To: sqlite-users@sqlite.org Subject: RE: [sqlite] Unlucky number for the ROUND function I reproduced it here as well. As near as I can tell, this line in printf.c (line 445): while( realvalue>10.0 && exp<=350 ){ realvalue *= 0.1; exp++; } should be: while( realvalue>=10.0 && exp<=350 ){ realvalue *= 0.1; exp++; } The comment at line 440 says it's trying to make realvalue be less than 10. Changing the line makes the round example given produce the right answer. The colon in the wrong answer is because et_getdigit is being passed 10.0 as the real value, and returing '0'+10 as the digit, which is ':'. --Ned. http://nedbatchelder.com -----Original Message----- From: Bob Dankert [mailto:[EMAIL PROTECTED] Sent: Tuesday, 30 August, 2005 2:35 PM To: sqlite-users@sqlite.org Subject: RE: [sqlite] Unlucky number for the ROUND function Using the sqlite.exe 3.2.5 binary off the website with WinXP SP2 here, I get the ":.0" result (less the quotes). I have tried this on a couple machines in the office here running similar environments. Bob Envision Information Technologies Associate [EMAIL PROTECTED] v. 608.256.5680 f. 608.256.3780 -----Original Message----- From: D. Richard Hipp [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 30, 2005 1:08 PM To: sqlite-users@sqlite.org Subject: RE: [sqlite] Unlucky number for the ROUND function On Tue, 2005-08-30 at 12:46 -0500, Bob Dankert wrote: > According to that, it rounds to the nearest even number. Shouldn't 9.95 > go to 10 then, and 9.85 go to 9.8? > > After additional testing with SQLite 3.2.2, I have the following > results: > > Round(9.95,1) -> 9.9 *Rounded Down* > Round(9.85,1) -> 9.8 *Rounded Down* > Round(9.5,0) -> 10 *Rounded Up* > Round(9.995,2) -> 9.99 *Rounded Down* > > I really see no pattern or sense to the results. > The reason you see no pattern is because you are thinking in decimal whereas SQLite thinks in binary. The number 9.95 does not a have finite representation in binary. The closest you can get with a 64-bit IEEE float is: 9.949999999999999289457264239899814128875732421875 So when you type "9.95" into an SQL statement, SQLite really inserts the number shown above, not 9.95. And the number shown above rounds down. 9.5 does have an exact representation in binary so it rounds as you would expect. But neither 9.85 nor 9.995 do - the binary values chosen to represent them are both just a little less than their decimal values. Hence they both round down. So I'm not overly worried when I see round(9.95,1) come out with 9.9. But I am concerned about the people who are seeing results like ":.0". I wish I could reproduce that problem. -- D. Richard Hipp <[EMAIL PROTECTED]>
