On 5/24/19 8:35 AM, Jose Isaias Cabrera wrote: > If IEEE754 can't figure out how to round 3.555 to 2 decimals, I don't know if > I'd trust it. ;-) thanks.
Just to point out a simple fact that seems to have been glossed over a bit. In IEEE754 there is NO number with a value of exactly 3.555 (or 3.56), as neither of these numbers can be expressed as an integer value times a power of 2. It defines that given these numbers as input requests, the conversion routine should generate one of the two numbers that can be exactly represented nearest to that value (one being something that begins 3.55499... and the other that begins 3.55500...), and preferably the one that is closest. Depending on which of these two numbers was chose to represent the value will affect which value the proper answer should be, which would be eith 3.55 or 3.56, and then you run into the issue that neither of THOSE values can be exactly represented, so the final answer will be the number either immediately below or above that value. Now, you also can't just go by what is printed for those numbers as usually the default format, even if you don't explicitly round, does some rounding to avoid a lot of 99999s being outputted. One way to show this issue is to do something like (3.555*1000)-3555, which if 3.555 could be exactly represented should be 0, but if the calculation is done in binary floating point, it won't be. (The 1000 and 3555, being reasonable sized integers can be exactly represented). This isn't so much a problem with IEEE754, but a fundamental problem with binary floating point and decimal numbers, which is why many business problems don't use floating point, but decimal arithmetic or scaling things to keep the important digits as integers (do you math in pennies or a fraction of a penny, so the binary issue isn't an issue any longer, since 0.5 is a number that IS exactly representable, so the rounding point is exact). -- Richard Damon _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
