https://bugs.documentfoundation.org/show_bug.cgi?id=157856
Mike Kaganski <mikekagan...@hotmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
See Also| |https://bugs.documentfounda
| |tion.org/show_bug.cgi?id=10
| |7953
--- Comment #3 from Mike Kaganski <mikekagan...@hotmail.com> ---
(In reply to cppjohn from comment #0)
> The “Sum:” value for “=0.95-0.9” on OpenOffice calc is 0.05 instead of
> 0.0499999999999999.
It is indeed unclear if this occurrence of "OpenOffice" in comment 0 actually
meant OpenOffice (better use AOO, to avoid confusion in the future), or was
meant to refer to LibreOffice (LO). But from context, it seems to mean AOO, as
showing "correct" answer; and so, bug 107953 seems relevant to *this specific
bit* related to display.
This faq has a general overview:
https://wiki.documentfoundation.org/Faq/Calc/Accuracy
It indeed not only relates to Calc, but to any floating-point processing. It
might happen, that we dropped some rounding, that allowed to compare some
different floating-point values equal; this would need checking (so please do
attach a sample ODS, with the macros; it is unclear what you did to get the
values from the cells - e.g., one could take strings, and convert to numbers;
however "clear" the textual description is - it is unreasonable to ask people
to re-invent the macros, and *hope* they have the same as you have).
This calculator allows to see the binary representations of doubles:
http://weitz.de/ieee/. It helps to see, that 0.05 + 0.90 results in
0.9500000000000001 (binary
0b0011111111101110011001100110011001100110011001100110011001100111), while 0.95
is 0b0011111110101001100110011001100110011001100110011001100110011010 (the
difference is in the last digit).
--
You are receiving this mail because:
You are the assignee for the bug.
