[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Hello Zoltan Borok-Nagy, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16741 to look at the new patch set (#10). Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions This patch implements Truncate and Bucket partition transforms in Impala BE as built-in functions. The expectation is that these functions give the same result as Iceberg's implementation of the same functions. These built-in functions are invisible so users won't be able to invoke them e.g. from impala-shell. Truncate: - Supported types are IntVal, BigIntVal, StringVal, DecimalVal. - Receives an input from the above types and a width. - Returns the same type as the input. - Expected behaviour is explained here: https://iceberg.apache.org/spec/#truncate-transform-details Bucket: - Supported types are IntVal, BigIntVal, StringVal, DecimalVal, DateVal, TimestampVal. - Receives an input from the above types and the number of buckets as IntVal. - Returns IntVal. - Expected behaviour is explained here: https://iceberg.apache.org/spec/#bucket-transform-details Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd --- M be/src/codegen/impala-ir.cc M be/src/exprs/CMakeLists.txt A be/src/exprs/iceberg-functions-ir.cc A be/src/exprs/iceberg-functions-test.cc A be/src/exprs/iceberg-functions.h M be/src/exprs/scalar-expr-evaluator.cc A be/src/thirdparty/murmurhash/MurmurHash3.cpp A be/src/thirdparty/murmurhash/MurmurHash3.h A be/src/thirdparty/murmurhash/README.md M be/src/util/bit-util-test.cc M be/src/util/bit-util.h M bin/rat_exclude_files.txt M bin/run_clang_tidy.sh M common/function-registry/impala_functions.py 14 files changed, 1,414 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/16741/10 -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 10 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16741 ) Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. Patch Set 9: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7839/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 9 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 11 Dec 2020 10:46:12 + Gerrit-HasComments: No
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16741 ) Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. Patch Set 9: (1 comment) http://gerrit.cloudera.org:8080/#/c/16741/8/be/src/thirdparty/murmurhash/MurmurHash3.h File be/src/thirdparty/murmurhash/MurmurHash3.h: http://gerrit.cloudera.org:8080/#/c/16741/8/be/src/thirdparty/murmurhash/MurmurHash3.h@24 PS8, Line 24: #else // defined(_MSC_VER) > tab used for whitespace Done -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 9 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 11 Dec 2020 10:39:38 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16741 ) Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7838/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 8 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 11 Dec 2020 10:32:07 + Gerrit-HasComments: No
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Hello Zoltan Borok-Nagy, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16741 to look at the new patch set (#9). Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions This patch implements Truncate and Bucket partition transforms in Impala BE as built-in functions. The expectation is that these functions give the same result as Iceberg's implementation of the same functions. These built-in functions are invisible so users won't be able to invoke them e.g. from impala-shell. Truncate: - Supported types are IntVal, BigIntVal, StringVal, DecimalVal. - Receives an input from the above types and a width. - Returns the same type as the input. - Expected behaviour is explained here: https://iceberg.apache.org/spec/#truncate-transform-details Bucket: - Supported types are IntVal, BigIntVal, StringVal, DecimalVal, DateVal, TimestampVal. - Receives an input from the above types and the number of buckets as IntVal. - Returns IntVal. - Expected behaviour is explained here: https://iceberg.apache.org/spec/#bucket-transform-details TODO gaborkaszab: This patch is WiP because Decimal support of Bucket transform is not part of this patch and to be implemented. Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd --- M be/src/codegen/impala-ir.cc M be/src/exprs/CMakeLists.txt A be/src/exprs/iceberg-functions-ir.cc A be/src/exprs/iceberg-functions-test.cc A be/src/exprs/iceberg-functions.h M be/src/exprs/scalar-expr-evaluator.cc A be/src/thirdparty/murmurhash/MurmurHash3.cpp A be/src/thirdparty/murmurhash/MurmurHash3.h A be/src/thirdparty/murmurhash/README.md M bin/rat_exclude_files.txt M bin/run_clang_tidy.sh M common/function-registry/impala_functions.py 12 files changed, 1,217 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/16741/9 -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 9 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16741 ) Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. Patch Set 8: (1 comment) http://gerrit.cloudera.org:8080/#/c/16741/7/be/src/exprs/iceberg-functions-ir.cc File be/src/exprs/iceberg-functions-ir.cc: http://gerrit.cloudera.org:8080/#/c/16741/7/be/src/exprs/iceberg-functions-ir.cc@59 PS7, Line 59: return TruncateDecimal(ctx, input, width, decimal_size); : } : > Could you add a comment what happens here? Shouldn't we use something like Actually this check is not needed as it is not a valid scenario to overflow here. The reason is that when an input is close to int32_t::min then the decimal is already represented on 8bytes instead of 4. -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 8 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 11 Dec 2020 10:15:49 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16741 ) Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. Patch Set 8: (1 comment) http://gerrit.cloudera.org:8080/#/c/16741/8/be/src/thirdparty/murmurhash/MurmurHash3.h File be/src/thirdparty/murmurhash/MurmurHash3.h: http://gerrit.cloudera.org:8080/#/c/16741/8/be/src/thirdparty/murmurhash/MurmurHash3.h@24 PS8, Line 24: #else // defined(_MSC_VER) tab used for whitespace -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 8 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 11 Dec 2020 10:10:59 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Hello Zoltan Borok-Nagy, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16741 to look at the new patch set (#8). Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions This patch implements Truncate and Bucket partition transforms in Impala BE as built-in functions. The expectation is that these functions give the same result as Iceberg's implementation of the same functions. These built-in functions are invisible so users won't be able to invoke them e.g. from impala-shell. Truncate: - Supported types are IntVal, BigIntVal, StringVal, DecimalVal. - Receives an input from the above types and a width. - Returns the same type as the input. - Expected behaviour is explained here: https://iceberg.apache.org/spec/#truncate-transform-details Bucket: - Supported types are IntVal, BigIntVal, StringVal, DecimalVal, DateVal, TimestampVal. - Receives an input from the above types and the number of buckets as IntVal. - Returns IntVal. - Expected behaviour is explained here: https://iceberg.apache.org/spec/#bucket-transform-details TODO gaborkaszab: This patch is WiP because Decimal support of Bucket transform is not part of this patch and to be implemented. Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd --- M be/src/codegen/impala-ir.cc M be/src/exprs/CMakeLists.txt A be/src/exprs/iceberg-functions-ir.cc A be/src/exprs/iceberg-functions-test.cc A be/src/exprs/iceberg-functions.h M be/src/exprs/scalar-expr-evaluator.cc A be/src/thirdparty/murmurhash/MurmurHash3.cpp A be/src/thirdparty/murmurhash/MurmurHash3.h A be/src/thirdparty/murmurhash/README.md M bin/rat_exclude_files.txt M bin/run_clang_tidy.sh M common/function-registry/impala_functions.py 12 files changed, 1,217 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/16741/8 -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 8 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Zoltan Borok-Nagy has posted comments on this change. (
http://gerrit.cloudera.org:8080/16741 )
Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition
transforms as built-in functions
..
Patch Set 7:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/16741/7/be/src/exprs/iceberg-functions-ir.cc
File be/src/exprs/iceberg-functions-ir.cc:
http://gerrit.cloudera.org:8080/#/c/16741/7/be/src/exprs/iceberg-functions-ir.cc@59
PS7, Line 59: if (input.val4 < 0 && result.val4 > 0) {
: return
TruncatePartitionTransformDecimalImpl(input.val4, width.val);
: }
Could you add a comment what happens here? Shouldn't we use something like
RETURN_IF_OVERFLOW in decimal-operators-ir.cc?
impala_udf::DecimalVal is able to hold decimals with any size, but
impala::DecimalVal might only have 4 bytes of storage, this might be
problematic in some cases.
--
To view, visit http://gerrit.cloudera.org:8080/16741
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd
Gerrit-Change-Number: 16741
Gerrit-PatchSet: 7
Gerrit-Owner: Gabor Kaszab
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Fri, 04 Dec 2020 11:48:30 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16741 ) Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7768/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 6 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 03 Dec 2020 09:43:16 + Gerrit-HasComments: No
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16741 ) Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/16741/6/be/src/thirdparty/murmurhash/MurmurHash3.h File be/src/thirdparty/murmurhash/MurmurHash3.h: http://gerrit.cloudera.org:8080/#/c/16741/6/be/src/thirdparty/murmurhash/MurmurHash3.h@21 PS6, Line 21: #else // defined(_MSC_VER) tab used for whitespace -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 6 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 03 Dec 2020 09:21:59 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Hello Zoltan Borok-Nagy, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16741 to look at the new patch set (#6). Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd --- M be/src/codegen/impala-ir.cc M be/src/exprs/CMakeLists.txt A be/src/exprs/iceberg-functions-ir.cc A be/src/exprs/iceberg-functions-test.cc A be/src/exprs/iceberg-functions.h M be/src/exprs/scalar-expr-evaluator.cc A be/src/thirdparty/murmurhash/MurmurHash3.cpp A be/src/thirdparty/murmurhash/MurmurHash3.h A be/src/thirdparty/murmurhash/README.md M bin/rat_exclude_files.txt M bin/run_clang_tidy.sh M common/function-registry/impala_functions.py 12 files changed, 860 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/16741/6 -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 6 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16741 ) Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7763/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 5 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 02 Dec 2020 08:56:54 + Gerrit-HasComments: No
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16741 ) Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. Patch Set 5: (4 comments) http://gerrit.cloudera.org:8080/#/c/16741/3/be/src/exprs/CMakeLists.txt File be/src/exprs/CMakeLists.txt: http://gerrit.cloudera.org:8080/#/c/16741/3/be/src/exprs/CMakeLists.txt@90 PS3, Line 90: A > nit: space Done http://gerrit.cloudera.org:8080/#/c/16741/3/be/src/exprs/iceberg-functions-ir.cc File be/src/exprs/iceberg-functions-ir.cc: http://gerrit.cloudera.org:8080/#/c/16741/3/be/src/exprs/iceberg-functions-ir.cc@26 PS3, Line 26: > nit: too much spaces Done http://gerrit.cloudera.org:8080/#/c/16741/3/be/src/exprs/iceberg-functions-ir.cc@77 PS3, Line 77: } > maybe we could have a fast-path when input.val is nonnegative, i.e. in that Sure. I added this and the same for decimals. http://gerrit.cloudera.org:8080/#/c/16741/3/be/src/exprs/iceberg-functions-ir.cc@84 PS3, Line 84: n StringVal::null(); > Is this correct formula if decimal_val is negative? Changed it meanwhile in PS4 -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 5 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 02 Dec 2020 08:35:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Hello Zoltan Borok-Nagy, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16741 to look at the new patch set (#5). Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd --- M be/src/codegen/impala-ir.cc M be/src/exprs/CMakeLists.txt A be/src/exprs/iceberg-functions-ir.cc A be/src/exprs/iceberg-functions-test.cc A be/src/exprs/iceberg-functions.h M be/src/exprs/scalar-expr-evaluator.cc M common/function-registry/impala_functions.py 7 files changed, 464 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/16741/5 -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 5 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16741 ) Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7760/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 4 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 01 Dec 2020 15:44:56 + Gerrit-HasComments: No
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Hello Zoltan Borok-Nagy, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16741 to look at the new patch set (#4). Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd --- M be/src/codegen/impala-ir.cc M be/src/exprs/CMakeLists.txt A be/src/exprs/iceberg-functions-ir.cc A be/src/exprs/iceberg-functions-test.cc A be/src/exprs/iceberg-functions.h M be/src/exprs/scalar-expr-evaluator.cc M common/function-registry/impala_functions.py 7 files changed, 462 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/16741/4 -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 4 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/16741 ) Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. Patch Set 3: (4 comments) http://gerrit.cloudera.org:8080/#/c/16741/3/be/src/exprs/CMakeLists.txt File be/src/exprs/CMakeLists.txt: http://gerrit.cloudera.org:8080/#/c/16741/3/be/src/exprs/CMakeLists.txt@90 PS3, Line 90: nit: space http://gerrit.cloudera.org:8080/#/c/16741/3/be/src/exprs/iceberg-functions-ir.cc File be/src/exprs/iceberg-functions-ir.cc: http://gerrit.cloudera.org:8080/#/c/16741/3/be/src/exprs/iceberg-functions-ir.cc@26 PS3, Line 26: nit: too much spaces http://gerrit.cloudera.org:8080/#/c/16741/3/be/src/exprs/iceberg-functions-ir.cc@77 PS3, Line 77: return input.val - (((input.val % width.val) + width.val) % width.val); maybe we could have a fast-path when input.val is nonnegative, i.e. in that case the expression could be simply: input.val - (input.val % width.val) http://gerrit.cloudera.org:8080/#/c/16741/3/be/src/exprs/iceberg-functions-ir.cc@84 PS3, Line 84: decimal_val - (decimal_val % width); Is this correct formula if decimal_val is negative? -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 3 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 30 Nov 2020 11:59:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16741 ) Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7740/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 3 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 26 Nov 2020 16:32:22 + Gerrit-HasComments: No
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Hello Zoltan Borok-Nagy, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16741 to look at the new patch set (#3). Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd --- M be/src/codegen/impala-ir.cc M be/src/exprs/CMakeLists.txt A be/src/exprs/iceberg-functions-ir.cc A be/src/exprs/iceberg-functions-test.cc A be/src/exprs/iceberg-functions.h M be/src/exprs/scalar-expr-evaluator.cc M common/function-registry/impala_functions.py 7 files changed, 360 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/16741/3 -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 3 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/16741 ) Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. Patch Set 1: Thanks for working on this. The change looks great, though I became a bit unsure about whether we want to make these functions visible to the user. However, it's definitely useful during development. Maybe write C++ unit tests instead of e2e tests, and later we can decide the visibility of these functions. -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 1 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 19 Nov 2020 14:46:00 + Gerrit-HasComments: No
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16741 ) Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7672/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 1 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 18 Nov 2020 14:52:12 + Gerrit-HasComments: No
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16741 ) Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/16741/1/tests/query_test/test_iceberg.py File tests/query_test/test_iceberg.py: http://gerrit.cloudera.org:8080/#/c/16741/1/tests/query_test/test_iceberg.py@61 PS1, Line 61: b flake8: E501 line too long (95 > 90 characters) -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 1 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 18 Nov 2020 14:31:27 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions
Gabor Kaszab has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16741 Change subject: WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions .. WiP: IMPALA-10237: Support Bucket and Truncate partition transforms as built-in functions Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd --- M be/src/codegen/impala-ir.cc M be/src/exprs/CMakeLists.txt A be/src/exprs/iceberg-functions-ir.cc A be/src/exprs/iceberg-functions.h M be/src/exprs/scalar-expr-evaluator.cc M common/function-registry/impala_functions.py A testdata/workloads/functional-query/queries/QueryTest/iceberg-transform-functions.test M tests/query_test/test_iceberg.py 8 files changed, 137 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/16741/1 -- To view, visit http://gerrit.cloudera.org:8080/16741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I485680cf79d96d578dd8cfbfd554bec468fe84bd Gerrit-Change-Number: 16741 Gerrit-PatchSet: 1 Gerrit-Owner: Gabor Kaszab
