Hello guys, just a word that I find it very interesting to try to recover from the float lost precision at the rounding stage by looking for the shortest decimal fraction that rounds correctly back to the true binary value. I always spotted and insisted a few years ago that even with good coding (not to be assumed too lightly) rounding would not behave like in a decimal system and i would make OpenERP numbers awkward at best, but possibly illegal if some law forces you to adopt some specific rounding technique.
Going to decimal might be the solution. But If that can not be done (probably not in 6.1), I think that re-defining a rounding function that will round like a decimal system (assuming a limited precision) is something interesting. I've been surprise o see that Python 2.7 would use the same technique now. For instance if you write 0.1 in the interpreter it will write 0.1 while it would have written 0.10000000000000001 in previous Python versions as it was closer to the approximated binary value for 0.1. I've played a bit with the r2 method provided in https://bugs.launchpad.net/openobject-server/+bug/882036/comments/19 def r2(f, p=0): return round(f + cmp(f, 0.0) * (.1 / pow(10, p+2)), p) Overall it's interesting but I could spot a bug: it has a very limited precision. If you try r2(1.00499, 2) you get 1.01 instead of 1.00 After my very little test, you can increase the precision of it by putting more then 2 in p+2, like: def r2(f, p=0): return round(f + cmp(f, 0.0) * (.1 / pow(10, p+5)), p) This increases the precision apparently. I didn't bother computing what would be the maximal precision one could reach. Now some questions: Do one of you know if that approach is totally flawed? If yes, could you provide some counter example that given some limited entry precision would fail to round a 2 digits like a decimal system? @Cloves, do you have some law text that says we should use ROUND_HALF_EVEN ? I've always looked for that and never found anything. Even in France I couldn't find such a text law (except for the old Franc / Euro conversion I think; but hey it may become actual again ;-) ) Regards. On Wed, Nov 23, 2011 at 2:52 PM, Ferdinand @ Camptocamp < [email protected]> wrote: > may be this highlights the problem > http://www.bytereef.org/mpdecimal/index.html > > -- > You received this bug notification because you are a member of OpenERP > Committers, which is subscribed to OpenERP Server. > https://bugs.launchpad.net/bugs/882036 > > Title: > rounding error > > Status in OpenERP Server: > New > > Bug description: > Concerns 6.0 and trunk. > If you define a precision of 0.01, the rounding of 0.125 must be 0.13 and > not 0.12. The error is in the call of the format string "%.2f"%val which > introduces a mathematical error. The round function must be called to apply > the correct rounding before formatting the string. It should be: > "%.2f"%round(val*100)/100 > Fix class digits_change of class float in the server. BUT fix also the > gtk client AND web client as they all have that error (I let you find the > right line) > > To manage notifications about this bug go to: > https://bugs.launchpad.net/openobject-server/+bug/882036/+subscriptions > -- You received this bug notification because you are a member of C2C OERPScenario, which is subscribed to OpenERP Project Group. https://bugs.launchpad.net/bugs/882036 Title: rounding error Status in OpenERP Server: New Bug description: Concerns 6.0 and trunk. If you define a precision of 0.01, the rounding of 0.125 must be 0.13 and not 0.12. The error is in the call of the format string "%.2f"%val which introduces a mathematical error. The round function must be called to apply the correct rounding before formatting the string. It should be: "%.2f"%round(val*100)/100 Fix class digits_change of class float in the server. BUT fix also the gtk client AND web client as they all have that error (I let you find the right line) To manage notifications about this bug go to: https://bugs.launchpad.net/openobject-server/+bug/882036/+subscriptions _______________________________________________ Mailing list: https://launchpad.net/~c2c-oerpscenario Post to : [email protected] Unsubscribe : https://launchpad.net/~c2c-oerpscenario More help : https://help.launchpad.net/ListHelp
