sal/qa/rtl/math/test-rtl-math.cxx | 4 ++++
sal/rtl/math.cxx | 2 --
2 files changed, 4 insertions(+), 2 deletions(-)
New commits:
commit 9adffb1c7453aee77f9d52b1ed0aa99191a9c317
Author: Po-Yen Huang <jeff.hu...@ossii.com.tw>
AuthorDate: Tue Nov 7 13:01:27 2023 +0800
Commit: Mike Kaganski <mike.kagan...@collabora.com>
CommitDate: Tue Feb 27 11:00:24 2024 +0100
tdf#158190 Fix Calc ROUND in floating-point calculate result very close to
X.5
Location:
if (nDecPlaces == 0)
{
switch (eMode)
{
case rtl_math_RoundingMode_Corrected:
return std::round(fValue);
Because the functions are the same as the following
related codes, they are respectively:
if (nDecPlaces >= 0
&& (fValue >= 0x1p52
|| isRepresentableInteger(fValue)))
return fOrigValue;
as well as:
if (fValue < 0x1p52)
{
switch(eMode)
{
case rtl_math_RoundingMode_Corrected:
fValue = rtl::math::approxFloor(fValue + 0.5);
break;
:
:
:
Because some double-precision floating point
numbers cause std::round(fValue) and rtl::math::approxFloor(fValue + 0.5)
to produce different results.
For example, enter 10614.4999999999876 in Calc's
A1 cell (or any operation that will produce a long
decimal number such as .499999999999xyz after the
decimal point).
We expected it to be 10614.4999999999876, but it
was automatically rounded to 10614.5 (note: this
result is also the result of ubiquitous handheld
computers, OpenOffice.org and Excel)
Then, entering =ROUND(A1,0) in B1, we will see 10614,
A1 and B1 clearly don't meet expectations. (My boss
or tax collector might be unhappy :-p)
Although A1's 10614.4999999999876 is still faithfully
recorded, the rendering of 10614.5 is confusing.
Now, there are two views:
1. According to the established convention, B2
displays the expected answer 10615.
or
2. True to the laws of mathematics, A1 displays
10614.4999999999876 or at least 10614.49
Although the second point of view is correct (and I
completely agree with it), when opening
spreadsheets generated by other software (such as
OpenOffice.org or Excel), the results will be different,
which people do not like to see.
So when nDecPlaces == 0 is removed, use
std::round(fValue) and let the existing code below it do the rounding:
if (fValue < 0x1p52)
{
switch(eMode)
{
case rtl_math_RoundingMode_Corrected:
fValue = rtl::math::approxFloor(fValue + 0.5);
This is consistent with the first point.
By the way, in nDecPlaces == 0
case rtl_math_RoundingMode_HalfEven:
This piece of code can be checked to see if it is really
necessary, because rtl_math_round() also has the same purpose code.
If it is not needed, the entire nDecPlaces == 0 can be removed.
Co-authored-by: Firefly <fire...@ossii.com.tw>
Co-authored-by: Franklin Weng <frank...@goodhorse.idv.tw>
Change-Id: If32cdb18c70ec0025c83ba25a99e5d135d66aec9
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/159193
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kagan...@collabora.com>
diff --git a/sal/qa/rtl/math/test-rtl-math.cxx
b/sal/qa/rtl/math/test-rtl-math.cxx
index bab2774a414c..a5ce3b4e6262 100644
--- a/sal/qa/rtl/math/test-rtl-math.cxx
+++ b/sal/qa/rtl/math/test-rtl-math.cxx
@@ -301,6 +301,10 @@ public:
fVal = 4503599627370491.0;
CPPUNIT_ASSERT_EQUAL( 4503599627370000.0, rtl::math::round( fVal, -3,
rtl_math_RoundingMode_Corrected));
+
+ // test #tdf158190: ROUND(16.83 * 650, 0)
+ fVal = 16.83;
+ CPPUNIT_ASSERT_EQUAL( 10940.0, rtl::math::round( fVal * 650, 0,
rtl_math_RoundingMode_Corrected));
}
void test_doubleToString() {
diff --git a/sal/rtl/math.cxx b/sal/rtl/math.cxx
index 68068eaf979f..75cada0b7d48 100644
--- a/sal/rtl/math.cxx
+++ b/sal/rtl/math.cxx
@@ -463,8 +463,6 @@ double SAL_CALL rtl_math_round(double fValue, int
nDecPlaces, enum rtl_math_Roun
{
switch (eMode)
{
- case rtl_math_RoundingMode_Corrected:
- return std::round(fValue);
case rtl_math_RoundingMode_HalfEven:
if (const int oldMode = std::fegetround();
std::fesetround(FE_TONEAREST) == 0)
{
