https://bugs.documentfoundation.org/show_bug.cgi?id=138360
Eike Rathke <er...@redhat.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Assignee|libreoffice-b...@lists.free |er...@redhat.com
|desktop.org |
Status|NEW |ASSIGNED
Summary|calculation: basic math |ROUND() large numbers to
|fail, !calc rounding wrong! |decimals and to negative -5
|dividing by inverse |decimals inaccurate
|sometimes different from |
|multiply, 7E10/1E-5 wrong, |
--- Comment #5 from Eike Rathke <er...@redhat.com> ---
ROUND(1/0.00001;-5)
0.00001 == 1e-5 => 1.0000000000000001e-05
1/0.00001 => 99999.999999999985
1/0.00001 * 1e-5 => 0.99999999999999989
+ 0.5 + ... (correction value) => 1.5000000000000091
floor() => 1
/ 1.0000000000000001e-05 => 99999.999999999985
In round(), using the inverse factor and swapping multiplication and division
gives the proper result:
1/0.00001 / 1e5 => 0.99999999999999989
+ 0.5 + 9e-15 (correction value) => 1.5000000000000091
floor() => 1
* 1e5 => 100000
For large values like ROUND(5e15+1;9) it could be determined if the value can
be rounded at all.
--
You are receiving this mail because:
You are the assignee for the bug._______________________________________________
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs
