https://bugs.documentfoundation.org/show_bug.cgi?id=168673

--- Comment #3 from Mike Kaganski <[email protected]> ---
Hmm.

The problem with inequality itself seems NOTABUG, and related to the usual
floating-point calculation imprecision [1]. Specifically:

1. Entering '01:26,47' time converts to 0,0010008101851851851;
2. In accordance to widely accepted policy of spreadsheets, when showing the
value, only 15 significands are shown: 0,00100081018518519;
3. The same number of significands is used to convert to strings, so the
'"="&A11' produces a Criterion "=0,00100081018518519";
4. When converting the "0,00100081018518519" to a number, the result is
0.0010008101851851901;
5. the comparison of these values in steps 1 and 4 produces false, even with
our normal approximate equality (the difference is ~5E-18, which is greater
than ~1E-18, which is the threshold used for approximate equality, calculated
as the original values * 1E-15).

I am not sure if we can improve this specific corner case - the usual "for any
floating-point calculations, use rounding" seems not fitting this well, but I
don't see a robust way out here. Increase threshold to original value * 5E-15 ?
I know for sure, that I tested 1E-14, and it was problematic; but I didn't test
5E-15.

===

However, the problem with subtraction, noticed in comment 1, is separate and
different. The formula

=A11-VALUE(TEXT(A11;"Standard"))

indeed produces -2,31481481481481E-14; whereas the formula

=RAWSUBTRACT(A1;VALUE(TEXT(A1;"General")))

produces the expected -4,98732999343332E-18.

Needs more investigation.

[1] https://wiki.documentfoundation.org/Faq/Calc/Accuracy

-- 
You are receiving this mail because:
You are the assignee for the bug.

Reply via email to