https://bugs.documentfoundation.org/show_bug.cgi?id=101837
--- Comment #15 from Aron Budea <ba...@caesar.elte.hu> ---
Very peculiar, in both cases the first 13 iterations of the while loop in
lcl_IterateInverse(...) ("inverse quadric interpolation with additional
brackets") are the exact same.
Then in the 14th iteration, this diverges:
fRy = rFunction.GetValue(fSx); <= rfunction is ScChiDistFunction
fRy becomes:
- in the good case: 0
- in the bad case: -4.6945954068622342e-17
However, in both cases the result of ScChiDistFunction::GetValue(...) is
supposed to be:
0.99999999999999911 - 0.99999999999999911
(return fp - rInt.GetChiDist(x, fDF);)
Surely the displayed value doesn't reflect the actual double value.
In the good case, the while loop exits afterwards, because this part of the
condition is false: fabs(fRy) > fYEps
In the bad case the while loop does an extra 3 iterations, which results in the
final discrepancy.
For the record, in the bad case the equality check only passes with 1 digit of
rounding, ie. ROUNDSIG(A8;1)
--
You are receiving this mail because:
You are the assignee for the bug._______________________________________________
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs
