https://bugs.documentfoundation.org/show_bug.cgi?id=168673
--- Comment #5 from Mike Kaganski <[email protected]> --- (In reply to Mike Kaganski from comment #3) > 5. the comparison of these values in steps 1 and 4 produces false, even with > our normal approximate equality (the difference is ~5E-18, which is greater > than ~1E-18, which is the threshold used for approximate equality, > calculated as the original values * 1E-15). > > I am not sure if we can improve this specific corner case - the usual "for > any floating-point calculations, use rounding" seems not fitting this well, > but I don't see a robust way out here. Increase threshold to original value > * 5E-15 ? I know for sure, that I tested 1E-14, and it was problematic; but > I didn't test 5E-15. FTR: the threshold is calculated using 0x1p-48 [1], which is ~3.5E-15; so it is greater than 1E-15, but less than 5E-15, which would be necessary for this case. Excel 2016 gives the expected 0 for the difference. [1] https://opengrok.libreoffice.org/xref/core/sal/rtl/math.cxx?r=c5c8e28959cc379452b15aa250ca7151a9e3d370&mo=23757&fi=604#604 -- You are receiving this mail because: You are the assignee for the bug.
