Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17303 )
Change subject: IMPALA-10654: Fix precision loss in DecimalValue to double conversion. ...................................................................... IMPALA-10654: Fix precision loss in DecimalValue to double conversion. Original approach to convert DecimalValue(internal representation of decimals) to double was not accurate. It was: static_cast<double>(value_) / pow(10.0, scale). However only integers from −2^53 to 2^53 can be represented accurately by double precision without any loss. Hence, it would not work for numbers like -0.43149576573887316. For DecimalValue representing -0.43149576573887316, value_ would be -43149576573887316 and scale would be 17. As value_ < -2^53, result would not be accurate. In newer approach we are using third party library https://github.com/lemire/fast_double_parser, which handles above scenario in a performant manner. Testing: 1. Added End to End Tests covering following scenarios: a. Test to show precision limitation of 16 in the write path b. DecimalValue's value_ between -2^53 and 2^53. b. value_ outside above range but abs(value_) < UINT64_MAX c. abs(value_) > UINT64_MAX -covers DecimalValue<__int128_t> 2. Ran existing backend and end-to-end tests completely Change-Id: I56f0652cb8f81a491b87d9b108a94c00ae6c99a1 Reviewed-on: http://gerrit.cloudera.org:8080/17303 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M be/src/runtime/decimal-value.inline.h A be/src/thirdparty/fast_double_parser/LICENSE A be/src/thirdparty/fast_double_parser/LICENSE.BSL A be/src/thirdparty/fast_double_parser/README.md A be/src/thirdparty/fast_double_parser/fast_double_parser.h M bin/rat_exclude_files.txt M testdata/workloads/functional-query/queries/QueryTest/values.test M tests/query_test/test_insert_parquet.py 8 files changed, 1,579 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17303 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I56f0652cb8f81a491b87d9b108a94c00ae6c99a1 Gerrit-Change-Number: 17303 Gerrit-PatchSet: 8 Gerrit-Owner: Amogh Margoor <amarg...@gmail.com> Gerrit-Reviewer: Amogh Margoor <amarg...@gmail.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>