Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11282 )
Change subject: IMPALA-7412: width_bucket() function overflows too easily ...................................................................... IMPALA-7412: width_bucket() function overflows too easily Running the tests of https://gerrit.cloudera.org/#/c/10859/ it turned out that the width_bucket() function overflows very often. A common problem is that the function tries to cast the 'num_buckets' parameter to the decimal determined by the Frontend. When the Frontend determined the precision and scale of this decimal it only considered the decimal arguments and ignored everything else. Therefore the determined precision and scale is often not suitable for the 'num_buckets' parameter. WidthBucketImpl() has three decimal arguments, all of them have the same byte size, precision, and scale. So it is possible to interpret them as plain integers and still calculate the proper bucket. I included the python test cases from IMPALA-7202 developed by Taras Bobrovytsky. I also extended the backend tests with new test cases. For performance test I used the following query: SELECT sum(width_bucket(cast(l_orderkey AS DECIMAL(30, 10)), 0, 5500000, 1000000)) FROM tpch_parquet.lineitem; The new implementation executed it in ~0.3 seconds. The old implementation executed it in ~0.8 seconds. Change-Id: I68262698144029ef7f54e027e586eaf105f36ab3 Reviewed-on: http://gerrit.cloudera.org:8080/11282 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M be/src/exprs/expr-test.cc M be/src/exprs/math-functions-ir.cc M be/src/util/bit-util.h M tests/query_test/test_decimal_fuzz.py 4 files changed, 200 insertions(+), 124 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11282 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I68262698144029ef7f54e027e586eaf105f36ab3 Gerrit-Change-Number: 11282 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Reviewer: anujphadke <apha...@cloudera.com>
