[Impala-ASF-CR] WIP IMPALA-12093: impala-shell should preserver all cookies by default
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19827 ) Change subject: WIP IMPALA-12093: impala-shell should preserver all cookies by default .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/16374/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19827 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic81f790288460b086ab218e6701e8115a996dfa7 Gerrit-Change-Number: 19827 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Mon, 17 Jun 2024 18:23:49 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-12093: impala-shell should preserver all cookies by default
Wenzhe Zhou has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19827 Change subject: WIP IMPALA-12093: impala-shell should preserver all cookies by default .. WIP IMPALA-12093: impala-shell should preserver all cookies by default Change-Id: Ic81f790288460b086ab218e6701e8115a996dfa7 --- M shell/ImpalaHttpClient.py M shell/cookie_util.py M shell/option_parser.py 3 files changed, 29 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/19827/2 -- To view, visit http://gerrit.cloudera.org:8080/19827 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic81f790288460b086ab218e6701e8115a996dfa7 Gerrit-Change-Number: 19827 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Michael Smith
[Impala-ASF-CR] WIP: IMPALA-13155: Smallify strings values in all Tuple::Deepcopy()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21508 ) Change subject: WIP: IMPALA-13155: Smallify strings values in all Tuple::Deepcopy() .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/16323/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21508 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5822243629e198e56006d81a9e5077926f391a74 Gerrit-Change-Number: 21508 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 12 Jun 2024 16:28:46 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-13155: Smallify strings values in all Tuple::Deepcopy()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21508 ) Change subject: WIP: IMPALA-13155: Smallify strings values in all Tuple::Deepcopy() .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/16322/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21508 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5822243629e198e56006d81a9e5077926f391a74 Gerrit-Change-Number: 21508 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 12 Jun 2024 16:15:55 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-13155: Smallify strings values in all Tuple::Deepcopy()
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21508 to look at the new patch set (#3). Change subject: WIP: IMPALA-13155: Smallify strings values in all Tuple::Deepcopy() .. WIP: IMPALA-13155: Smallify strings values in all Tuple::Deepcopy() Currently "Tuple::DeepCopy(const TupleDescriptor& desc, char** data, int* offset, bool convert_ptrs)" does not try to smallify string values, although it could safely do that as it creates new string values. We use that version of DeepCopy when we BROADCAST data between fragments, so smallifying on that path can be beneficial. TODO: measurements Change-Id: I5822243629e198e56006d81a9e5077926f391a74 --- M be/src/runtime/tuple.cc 1 file changed, 6 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/08/21508/3 -- To view, visit http://gerrit.cloudera.org:8080/21508 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5822243629e198e56006d81a9e5077926f391a74 Gerrit-Change-Number: 21508 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] WIP: IMPALA-13155: Smallify strings values in all Tuple::Deepcopy()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21508 ) Change subject: WIP: IMPALA-13155: Smallify strings values in all Tuple::Deepcopy() .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/16321/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21508 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5822243629e198e56006d81a9e5077926f391a74 Gerrit-Change-Number: 21508 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 12 Jun 2024 16:10:38 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu
Ashwani Raina has posted comments on this change. ( http://gerrit.cloudera.org:8080/21492 ) Change subject: WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/21492/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21492/2//COMMIT_MSG@16 PS2, Line 16: To be able to read back Kudu tables written by Impala correctly : convert_kudu_utc_timestamps and write_kudu_utc_timestamps need to : have the same value. > This leads to an obvious question: why to introduce an extra option --write +1 http://gerrit.cloudera.org:8080/#/c/21492/2/common/thrift/ImpalaService.thrift File common/thrift/ImpalaService.thrift: http://gerrit.cloudera.org:8080/#/c/21492/2/common/thrift/ImpalaService.thrift@946 PS2, Line 946: WRITE_KUDU_UTC_TIMESTAMPS Make both the query names similar. For read, it starts with CONVERT_ and for write it is simply WRITE_ Would be more uniformity in names, if it looks something like of these options: READ_CONVERT_KUDU_UTC_TIMESTAMPS / WRITE_CONVERT_KUDU_UTC_TIMESTAMPS OR READ_KUDU_UTC_TIMESTAMPS / WRITE_KUDU_UTC_TIMESTAMPS or anything you seem fit where naming convention is uniform. -- To view, visit http://gerrit.cloudera.org:8080/21492 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb4995a64e042e7bb261fcc6e6bf7ffce61e9bd1 Gerrit-Change-Number: 21492 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Mon, 10 Jun 2024 08:58:35 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/21492 ) Change subject: WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/21492/2/common/thrift/ImpalaService.thrift File common/thrift/ImpalaService.thrift: http://gerrit.cloudera.org:8080/#/c/21492/2/common/thrift/ImpalaService.thrift@882 PS2, Line 882: When true, TIMESTAMPs read from Kudu will be converted from UTC to local time. > While you are at this, I'd suggest stop using the deprecated TIMESTAMP name To be complete: When true, UNIXTIME_MICRO column values read from Kudu tables are converted into UTC timezone timestamps. Otherwise, UNIXTIME_MICRO column values are converted into impalad's timezone timestamps. -- To view, visit http://gerrit.cloudera.org:8080/21492 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb4995a64e042e7bb261fcc6e6bf7ffce61e9bd1 Gerrit-Change-Number: 21492 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Sat, 08 Jun 2024 00:31:44 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/21492 ) Change subject: WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu .. Patch Set 2: (7 comments) I took a quick look. I don't know much about Impala's planning phase and whether there are other places that need to be updated in this context, and my comments might be considered as nits/cosmetic stuff. However, I think it makes sense to make a bit of effort to reduce the confusion w.r.t. Kudu not actually supporting a timestamp type, but rather being Unix epoch time with microsecond precision. Thank you! http://gerrit.cloudera.org:8080/#/c/21492/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21492/2//COMMIT_MSG@12 PS2, Line 12: utc nit: UTC http://gerrit.cloudera.org:8080/#/c/21492/2//COMMIT_MSG@16 PS2, Line 16: To be able to read back Kudu tables written by Impala correctly : convert_kudu_utc_timestamps and write_kudu_utc_timestamps need to : have the same value. This leads to an obvious question: why to introduce an extra option --write_kudu_utc_timestamps then? Shouldn't Impala's behavior be updated to perform the conversion upon writing as well if --convert_kudu_utc_timestamps=true? If it's indeed necessary to separate two different phases of the conversion, maybe it makes sense to articulate why adding this new flag --write_kudu_utc_timestamps is needed? Mentioning a particular use case might be enough, I guess. http://gerrit.cloudera.org:8080/#/c/21492/2/common/thrift/ImpalaService.thrift File common/thrift/ImpalaService.thrift: http://gerrit.cloudera.org:8080/#/c/21492/2/common/thrift/ImpalaService.thrift@882 PS2, Line 882: When true, TIMESTAMPs read from Kudu will be converted from UTC to local time. While you are at this, I'd suggest stop using the deprecated TIMESTAMP name for the corresponding Kudu column type. It's been renamed into UNIXTIME_MICROS a long time ago -- that's to reduce the confusion. That reflects the fact that Kudu doesn't have a timestamp type in the common sense at least how it's been used in the database realm, but columns of the UNIXTIME_MICROS type are used to store Unix epoch time with microsecond precision, and no timezone information is provided along with 64-bit integer (so, technically that's not a timestamp in that sense). Maybe, this should be replaced with: When true, UNIXTIME_MICRO column values read from Kudu tables are converted into UTC timezone timestamps. http://gerrit.cloudera.org:8080/#/c/21492/2/common/thrift/ImpalaService.thrift@944 PS2, Line 944: When true, TIMESTAMPs written to Kudu will be converted from local time to UTC. Ditto: please consider updating this similar to the comment for CONVERT_KUDU_UTC_TIMESTAMPS above. http://gerrit.cloudera.org:8080/#/c/21492/2/testdata/workloads/functional-query/queries/QueryTest/kudu_timestamp_conversion.test File testdata/workloads/functional-query/queries/QueryTest/kudu_timestamp_conversion.test: http://gerrit.cloudera.org:8080/#/c/21492/2/testdata/workloads/functional-query/queries/QueryTest/kudu_timestamp_conversion.test@a3 PS2, Line 3: what's 'kutu'? http://gerrit.cloudera.org:8080/#/c/21492/2/testdata/workloads/functional-query/queries/QueryTest/kudu_timestamp_conversion.test@3 PS2, Line 3: UTC timestamp Please consider updating this -- Kudu doesn't store timestamps, it stores Unix epoch times. http://gerrit.cloudera.org:8080/#/c/21492/2/tests/common/test_result_verifier.py File tests/common/test_result_verifier.py: http://gerrit.cloudera.org:8080/#/c/21492/2/tests/common/test_result_verifier.py@427 PS2, Line 427: if file_format == 'avro' and 'TIMESTAMP' in expected_types: I don't know much about the context here, but Kudu still doesn't support TIMESTAMP type in the sense of storing date/time along with timezone information. In Kudu that's just Unix epoch times with microsecond precision. -- To view, visit http://gerrit.cloudera.org:8080/21492 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb4995a64e042e7bb261fcc6e6bf7ffce61e9bd1 Gerrit-Change-Number: 21492 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Sat, 08 Jun 2024 00:27:08 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21492 ) Change subject: WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu .. Patch Set 1: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/10695/ -- To view, visit http://gerrit.cloudera.org:8080/21492 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb4995a64e042e7bb261fcc6e6bf7ffce61e9bd1 Gerrit-Change-Number: 21492 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Fri, 07 Jun 2024 21:13:35 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21492 ) Change subject: WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/16296/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21492 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb4995a64e042e7bb261fcc6e6bf7ffce61e9bd1 Gerrit-Change-Number: 21492 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Fri, 07 Jun 2024 18:10:14 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu
Hello Zihao Ye, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21492 to look at the new patch set (#2). Change subject: WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu .. WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu Before this commit only read support was implemented (convert_kudu_utc_timestamps=true). This change adds write support: if write_kudu_utc_timestamps=true, then timestamps are converted from local time to utc during INSERT/UPSERT to Kudu. In case of ambigious conversions (DST changes) the earlier possible UTC timestamp is written. To be able to read back Kudu tables written by Impala correctly convert_kudu_utc_timestamps and write_kudu_utc_timestamps need to have the same value. The conversion is implemented by adding to_utc_timestamp() to inserted timestamp expressions during planning. This allows doing the same conversion during the pre-insert sorting and partitioning. Read support is implemented differently - in that case the plan is not changed and the scanner does the conversion. Other changes: - Before this change, verification of tests with TIMESTAMP results were skipped when the file format is Kudu. This shouldn't be necessary so the skipping was removed. Haven't run the full test suite yet, for this reason the change is WIP. Change-Id: Ibb4995a64e042e7bb261fcc6e6bf7ffce61e9bd1 --- M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaService.thrift M common/thrift/Query.thrift M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/util/ExprUtil.java M testdata/workloads/functional-query/queries/QueryTest/kudu_predicate_with_timestamp_conversion.test M testdata/workloads/functional-query/queries/QueryTest/kudu_runtime_filter_with_timestamp_conversion.test M testdata/workloads/functional-query/queries/QueryTest/kudu_timestamp_conversion.test M tests/common/test_result_verifier.py M tests/query_test/test_kudu.py 11 files changed, 95 insertions(+), 32 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/21492/2 -- To view, visit http://gerrit.cloudera.org:8080/21492 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibb4995a64e042e7bb261fcc6e6bf7ffce61e9bd1 Gerrit-Change-Number: 21492 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zihao Ye
[Impala-ASF-CR] WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21492 ) Change subject: WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/16293/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21492 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb4995a64e042e7bb261fcc6e6bf7ffce61e9bd1 Gerrit-Change-Number: 21492 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Fri, 07 Jun 2024 16:36:02 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21492 ) Change subject: WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10695/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/21492 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb4995a64e042e7bb261fcc6e6bf7ffce61e9bd1 Gerrit-Change-Number: 21492 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 07 Jun 2024 16:12:02 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu
Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21492 Change subject: WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu .. WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu Before this commit only read support was implemented (convert_kudu_utc_timestamps=true). This change adds write support: if write_kudu_utc_timestamps=true, then timestamps are converted from local time to utc during INSERT/UPSERT to Kudu. In case of ambigious conversions (DST changes) the earlier possible UTC timestamp is written. To be able to read back Kudu tables written by Impala correctly convert_kudu_utc_timestamps and write_kudu_utc_timestamps need to have the same value. The conversion is implemented by adding to_utc_timestamp() to inserted timestamp expressions during planning. This allows doing the same conversion during the pre-insert sorting and partitioning. Read support is implemented differently - in that case the plan is not changed and the scanner does the conversion. Other changes: - Before this change, verification of tests with TIMESTAMP results were skipped when the file format is Kudu. This shouldn't be necessary so the skipping was removed. Haven't run the full test suite yet, for this reason the change is WIP. Change-Id: Ibb4995a64e042e7bb261fcc6e6bf7ffce61e9bd1 --- M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaService.thrift M common/thrift/Query.thrift M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/util/ExprUtil.java M testdata/workloads/functional-query/queries/QueryTest/kudu_timestamp_conversion.test M tests/common/test_result_verifier.py M tests/query_test/test_kudu.py 9 files changed, 86 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/21492/1 -- To view, visit http://gerrit.cloudera.org:8080/21492 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ibb4995a64e042e7bb261fcc6e6bf7ffce61e9bd1 Gerrit-Change-Number: 21492 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer
[Impala-ASF-CR] WIP: IMPALA-13140: Add backend flag to disable small string optimization
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21489 ) Change subject: WIP: IMPALA-13140: Add backend flag to disable small string optimization .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/16291/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21489 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I71c590e5d93a638192f8e5c0b649b469d5f4ffc9 Gerrit-Change-Number: 21489 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 07 Jun 2024 14:17:29 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-13140: Add backend flag to disable small string optimization
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21489 to look at the new patch set (#4). Change subject: WIP: IMPALA-13140: Add backend flag to disable small string optimization .. WIP: IMPALA-13140: Add backend flag to disable small string optimization This patch adds a backend flag that makes SmallableString::Smallify() a no-op. Smallify() can be problematic when we are modifying existing StringValues of existing tuples. TODO: Testing Change-Id: I71c590e5d93a638192f8e5c0b649b469d5f4ffc9 --- M be/src/runtime/CMakeLists.txt A be/src/runtime/smallable-string.cc M be/src/runtime/smallable-string.h 3 files changed, 28 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/21489/4 -- To view, visit http://gerrit.cloudera.org:8080/21489 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I71c590e5d93a638192f8e5c0b649b469d5f4ffc9 Gerrit-Change-Number: 21489 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] WIP: IMPALA-13140: Add backend flag to disable small string optimization
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21489 ) Change subject: WIP: IMPALA-13140: Add backend flag to disable small string optimization .. Patch Set 3: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/16290/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/21489 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I71c590e5d93a638192f8e5c0b649b469d5f4ffc9 Gerrit-Change-Number: 21489 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 07 Jun 2024 13:04:48 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-13140: Add backend flag to disable small string optimization
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21489 to look at the new patch set (#3). Change subject: WIP: IMPALA-13140: Add backend flag to disable small string optimization .. WIP: IMPALA-13140: Add backend flag to disable small string optimization This patch adds a backend flag that makes SmallableString::Smallify() a no-op. Smallify() can be problematic when we are modifying existing StringValues of existing tuples. TODO: Testing Change-Id: I71c590e5d93a638192f8e5c0b649b469d5f4ffc9 --- M be/src/runtime/CMakeLists.txt M be/src/runtime/smallable-string.h 2 files changed, 7 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/21489/3 -- To view, visit http://gerrit.cloudera.org:8080/21489 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I71c590e5d93a638192f8e5c0b649b469d5f4ffc9 Gerrit-Change-Number: 21489 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] WIP: IMPALA-13140: Add backend flag to disable small string optimization
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21489 ) Change subject: WIP: IMPALA-13140: Add backend flag to disable small string optimization .. Patch Set 2: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/16289/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/21489 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I71c590e5d93a638192f8e5c0b649b469d5f4ffc9 Gerrit-Change-Number: 21489 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 07 Jun 2024 12:57:43 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-13140: Add backend flag to disable small string optimization
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21489 to look at the new patch set (#2). Change subject: WIP: IMPALA-13140: Add backend flag to disable small string optimization .. WIP: IMPALA-13140: Add backend flag to disable small string optimization This patch adds a backend flag that makes SmallableString::Smallify() a no-op. Smallify() can be problematic when we are modifying existing StringValues of existing tuples. TODO: Testing Change-Id: I71c590e5d93a638192f8e5c0b649b469d5f4ffc9 --- M be/src/runtime/smallable-string.h 1 file changed, 5 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/21489/2 -- To view, visit http://gerrit.cloudera.org:8080/21489 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I71c590e5d93a638192f8e5c0b649b469d5f4ffc9 Gerrit-Change-Number: 21489 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] WIP: IMPALA-13140: Add backend flag to disable small string optimization
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21489 ) Change subject: WIP: IMPALA-13140: Add backend flag to disable small string optimization .. Patch Set 1: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/16288/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/21489 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I71c590e5d93a638192f8e5c0b649b469d5f4ffc9 Gerrit-Change-Number: 21489 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 07 Jun 2024 08:56:42 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-13140: Add backend flag to disable small string optimization
Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21489 Change subject: WIP: IMPALA-13140: Add backend flag to disable small string optimization .. WIP: IMPALA-13140: Add backend flag to disable small string optimization This patch adds a backend flag that makes SmallableString::Smallify() a no-op. Smallify() can be problematic when we are modifying existing StringValues of existing tuples. Change-Id: I71c590e5d93a638192f8e5c0b649b469d5f4ffc9 TODO: Testing --- M be/src/runtime/smallable-string.h 1 file changed, 4 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/21489/1 -- To view, visit http://gerrit.cloudera.org:8080/21489 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I71c590e5d93a638192f8e5c0b649b469d5f4ffc9 Gerrit-Change-Number: 21489 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy
[Impala-ASF-CR] WIP
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21382 to look at the new patch set (#5). Change subject: WIP .. WIP IMPALA-12559: Support x5c Parameter in JSON Web Keys This enables the jwt verification using the x5c certificates in the jwks key. The x5c claim is part of the jwks either as a string or an array. This patch only supports one x5c certificate per public key. Testing: * added unit test case VerifyJwtTokenWithx5cCertificate to verify jwt with x5c certificate. Change-Id: I70be6f9f54190544aa005b2644e2ed8db6f6bb74 --- M be/src/util/jwt-util-test.cc M be/src/util/jwt-util.cc 2 files changed, 202 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/21382/5 -- To view, visit http://gerrit.cloudera.org:8080/21382 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I70be6f9f54190544aa005b2644e2ed8db6f6bb74 Gerrit-Change-Number: 21382 Gerrit-PatchSet: 5 Gerrit-Owner: gaurav singh Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] WIP
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21382 ) Change subject: WIP .. Patch Set 5: (3 comments) http://gerrit.cloudera.org:8080/#/c/21382/5/be/src/util/jwt-util-test.cc File be/src/util/jwt-util-test.cc: http://gerrit.cloudera.org:8080/#/c/21382/5/be/src/util/jwt-util-test.cc@385 PS5, Line 385: "MIIG1TCCBL2gAwIBAgIIFvMVGp6t\/cMwDQYJKoZIhvcNAQELBQAwZjELMAkGA1UEBhMCR0IxIDAeBgNVBAoMF1N0YW5kYXJkIENoYXJ0ZXJlZCBCYW5rMTUwMwYDVQQDDCxTdGFuZGFyZCBDaGFydGVyZWQgQmFuayBTaWduaW5nIENBIEcxIC0gU0hBMjAeFw0xODEwMTAxMTI2MzVaFw0yMjEwMTAxMTI2MzVaMIG9MQswCQYDVQQGEwJTRzESMBAGA1UECAwJU2luZ2Fwb3JlMRIwEAYDVQQHDAlTaW5nYXBvcmUxIDAeBgNVBAoMF1N0YW5kYXJkIENoYXJ0ZXJlZCBCYW5rMRwwGgYDVQQLDBNGb3VuZGF0aW9uIFNlcnZpY2VzMSgwJgYDVQQDDB9pbnRlcm5hbC1nYXRld2F5LWp3dC5hcGkuc2MubmV0MRwwGgYJKoZIhvcNAQkBFg1BUElQU1NAc2MuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArVWBoIi3IJ4nOWXu7\/SDxczqMou1B+c4c2FdQrOXrK31HxAaz4WEtma9BLXFdFHJ5mCCPIvdUcVxxnCynqhMOkZ\/a7acQbUD9cDzI8isMB9JL7VooDw0CctxHxffjqQQVIEhC2Q7zsM1pQayR7cl+pbBlvHIoRxq2n1B0fFvfoiosjf4kDiCpgHdM+v5Hw9aVYmUbroHxmQWqhB0iRTJQPPLZqqQVC50A1Q\/96gkwoODyotc46Uy9wYEpdGrtDG\/thWay3fmMsjpWR0U25xFIrxTrfCGBblYpD7juukWWml2E9rtE2rHgUxbymxXjEw7xrMwcGrhOGyqwoBqJy1JVwIDAQABo4ICLTCCAikwZAYIKwYBBQUHAQEEWDBWMFQGCCsGAQUFBzABhkhodHRwOi8vY29yZW9jc3AuZ2xvYmFsLnN0YW5kYXJkY2hhcnRlcmVkLmNvbS9lamJjYS9wdWJsaWN3ZWIvc3RhdHVzL29jc3AwHQYDVR0OBBYEFIinW4BNDeVEFcuLf8YjZjtySoW9MAwGA1UdEwEB\/wQCMAAwHwYDVR0jBBgwFoAUfNZMoZi33nKrcmVU3TFVQnuEi\/4wggFCBgNVHR8EggE5MIIBNTCCATGggcKggb+GgbxodHRwOi8vY29yZWNybC5nbG9iYWwuc3RhbmRhcmRjaGFydGVyZWQuY29tL2VqYmNhL3B1YmxpY3dlYi93ZWJkaXN0L2NlcnRkaXN0P2NtZD1jcmwmaXNzdWVyPUNOPVN0YW5kYXJkJTIwQ2hhcnRlcmVkJTIwQmFuayUyMFNpZ25pbmclMjBDQSUyMEcxJTIwLSUyMFNIQTIsTz1TdGFuZGFyZCUyMENoYXJ0ZXJlZCUyMEJhbmssQz1HQqJqpGgwZjE1MDMGA1UEAwwsU3RhbmRhcmQgQ2hhcnRlcmVkIEJhbmsgU2lnbmluZyBDQSBHMSAtIFNIQTIxIDAeBgNVBAoMF1N0YW5kYXJkIENoYXJ0ZXJlZCBCYW5rMQswCQYDVQQGEwJHQjAOBgNVHQ8BAf8EBAMCBsAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMA0GCSqGSIb3DQEBCwUAA4ICAQBtsoRlDHuOTDChcWdfdVUtRgP0U0ijDSeJi8vULN1rgYnqqJc4PdJno50aiu9MGlxY02O7HW7ZVD6QEG\/pqHmZ0sbWpb\/fumMgZSjP65IcGuS53zgcNtLYnyXyEv+v5T\/CK3bk4Li6tUW3ScJPUwVWwP1E0\/u6aBSb5k\/h4lTwS1o88ybS5pJOg6XutXByp991QQrrs7tp7fKNynjNZbFuG3J1e09X+zTfJOpjaDUofQTkt8IyMRI6Cs4wI1eZA+dAIL8B0n8ze1mRl1FOJqgdZrAQjoqZkCTnc0Il5VY\/dUXxGVg6D9e5pfck3FWT107K9\/5EZoxytpqYXFCjMXi5hx4YjK17OUgm82mZhvqkNdzF8Yq2vFuB3LPfyelESq99xFLykvinrVm1NtZKeDTT1Jq\/VvZt6stO\/tovq1RfJJcznpYcwOzxlnhGR6E+hxuBx7aDJzGf0JaoRxQILH1B2XV9WDI3HPYQsP7XtriX+QUJ\/aly28QkV48RmaGYCsly43YZu1MKudSsw+dhnbZzRsg\/aes3dzGW2x137bQPtux7k2LCSpsTXgedhOys28YoGlsoe8kUv0myAU4Stt+I3mrwO3BKUn+tJggvlDiiiyT1tg2HiklyU\/2FxQkZRMeB0eRrXTpg3l9x2mpF+dDFxOMKszxwD2kgoEZgo6o58A==", line too long (2378 > 90) http://gerrit.cloudera.org:8080/#/c/21382/5/be/src/util/jwt-util-test.cc@386 PS5, Line 386: "MIIE2jCCAsICAQEwDQYJKoZIhvcNAQELBQAwMzELMAkGA1UEBhMCVVMxEDAOBgNVBAoMB0pXVC1DUFAxEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0yMzEyMjIxMzIzNTdaFw0zMzEyMTkxMzIzNTdaMDMxCzAJBgNVBAYTAlVTMRAwDgYDVQQKDAdKV1QtQ1BQMRIwEAYDVQQDDAlsb2NhbGhvc3QwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDl0gyL9KCpoXsJlvBaBUsJTAgvFhbgsBjpxT6m2xv0fgtGDBwgaiPvuMnClTU/kYkKb8c1GTkMedKp9YcM57HWTk9yqGTy6QBnMMxbAJYNwWQ4Dbr4qKSC6C3KzYws/Bqyv8OC9NAOyqJbtdp4iObRjyaet+PLTXywuu02xtyRg3B+1aAONgUVDyS5u57NSD4rEZ+rw30Ne1doSClWmMDqEd72y8cjx3eAqn0HcAxSQ6MNMmNk7/M8FQD3DTM1Ef0G5oHyJIw7WmY+gxuD8386r/CkswINzadMwObPlTSdAN8BRzedtrqgb+D/K4pi2zhCiuIVujFX6M/hsGvj7g2M9E9MR8iEuHWCY9frQKIR+JTH3D1snoJp60qKoa51qBznsEr9RP2utGniPCq3+JY+ZX0JK8vl5tiSZpy6N0yRbRmY3XLdA5fKRzhcsB3eUrmTtr9ywjZX7Ll6QMvUyicubGTojhqJFQbvuvvops9PoCMXFE3x6cJ2QhPoi8+BvUdYisrtjDFe+YgrgQvPMa/CpOpDJJDEs2SVRcauCZOUdqLCwZylNuW0CgIjWP8l99P7l1zGeT8VJPhmABYyPM+RtNYDamAlUOCqRqgz/gPjEeMeulQTvH1lAqATAAX1oftlq6o4VoqROs2M3eAXqPhvsLBeTmCob+5ca887MkcP6wIDAQABMA0GCSqGSIb3DQEBCwUAA4ICAQBW2kREK4hlzxCDqykxrwfbQpiPwrbFmn+3RDJla+pI4L3wrvYT1nU96guFIU3zKnbMzqwPMRUCUjadr2jKxAmMWxCd/ThHQB+ne5xTvx7/6RVQfGjyMCG/SZtSH8/aO7ILNRtPT+SL5ZZwezaqv6gD89tSXB/w/0pYXy70wDuU17KCrTsKSISWGJ1cKi5l2R/m/ZaGjcV8U8NcFepF2bX3u/i0zhaqOqjiwrSEt7fWGDLabPs6n7GtfibZROEDZ/h0JrDINC+6mSfTOYAMJvGjeHA3H/NvzqR+CJgpXGCqElqVuBF0HdxPmwRRBoZC/BLIEcz0VHmB4rcpfaV47TZT+J+04fHYp4Y1S0u112CDrDe+61cDrnbDHC7aGX0G93pYSBKAB1e3LLc9rXQgf2F0pRtFB3rgZA9MtJ+TL7DUvY4VXJNq3v7UolIdldYRdk21YqAS2Hp0fivvFoEk2P/WbwDEErxR0FkZ/JQoI9FMJ9AvDxa4MsFFtlQVInfD2HUu+nhnuEAA8R6L+F2XqhfLY/H7H31iFBK6UCuqptED71VwWHqfBsAPRhLXAqGco7Ln2dzioyj0QdwJqQQIqigltSYtXxfIMLW0BekQ5yln7QTxnZlobkPHUW9s3NK+OMLuKCzVREzjic/aioQP3cRBMXkG2deMwrk3aX8yJuz4gA==" line too long (1682 > 90) http://gerrit.cloudera.org:8080/#/c/21382/5/be/src/util/jwt-util-test.cc@388 PS5, Line 388: "n":
[Impala-ASF-CR] WIP
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21382 ) Change subject: WIP .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/16076/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21382 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I70be6f9f54190544aa005b2644e2ed8db6f6bb74 Gerrit-Change-Number: 21382 Gerrit-PatchSet: 4 Gerrit-Owner: gaurav singh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 01 May 2024 19:00:11 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21382 to look at the new patch set (#4). Change subject: WIP .. WIP IMPALA-12559: Support x5c Parameter in JSON Web Keys This enables the jwt verification using the x5c certificates in the jwks key. The x5c claim is part of the jwks either as a string or an array. This patch only supports one x5c certificate per public key. Testing: * added unit test case VerifyJwtTokenWithx5cCertificate to verify jwt with x5c certificate. Change-Id: I70be6f9f54190544aa005b2644e2ed8db6f6bb74 --- M be/src/util/jwt-util-test.cc M be/src/util/jwt-util.cc 2 files changed, 191 insertions(+), 19 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/21382/4 -- To view, visit http://gerrit.cloudera.org:8080/21382 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I70be6f9f54190544aa005b2644e2ed8db6f6bb74 Gerrit-Change-Number: 21382 Gerrit-PatchSet: 4 Gerrit-Owner: gaurav singh Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] WIP
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21382 ) Change subject: WIP .. Patch Set 4: (3 comments) http://gerrit.cloudera.org:8080/#/c/21382/4/be/src/util/jwt-util-test.cc File be/src/util/jwt-util-test.cc: http://gerrit.cloudera.org:8080/#/c/21382/4/be/src/util/jwt-util-test.cc@385 PS4, Line 385: "MIIG1TCCBL2gAwIBAgIIFvMVGp6t\/cMwDQYJKoZIhvcNAQELBQAwZjELMAkGA1UEBhMCR0IxIDAeBgNVBAoMF1N0YW5kYXJkIENoYXJ0ZXJlZCBCYW5rMTUwMwYDVQQDDCxTdGFuZGFyZCBDaGFydGVyZWQgQmFuayBTaWduaW5nIENBIEcxIC0gU0hBMjAeFw0xODEwMTAxMTI2MzVaFw0yMjEwMTAxMTI2MzVaMIG9MQswCQYDVQQGEwJTRzESMBAGA1UECAwJU2luZ2Fwb3JlMRIwEAYDVQQHDAlTaW5nYXBvcmUxIDAeBgNVBAoMF1N0YW5kYXJkIENoYXJ0ZXJlZCBCYW5rMRwwGgYDVQQLDBNGb3VuZGF0aW9uIFNlcnZpY2VzMSgwJgYDVQQDDB9pbnRlcm5hbC1nYXRld2F5LWp3dC5hcGkuc2MubmV0MRwwGgYJKoZIhvcNAQkBFg1BUElQU1NAc2MuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArVWBoIi3IJ4nOWXu7\/SDxczqMou1B+c4c2FdQrOXrK31HxAaz4WEtma9BLXFdFHJ5mCCPIvdUcVxxnCynqhMOkZ\/a7acQbUD9cDzI8isMB9JL7VooDw0CctxHxffjqQQVIEhC2Q7zsM1pQayR7cl+pbBlvHIoRxq2n1B0fFvfoiosjf4kDiCpgHdM+v5Hw9aVYmUbroHxmQWqhB0iRTJQPPLZqqQVC50A1Q\/96gkwoODyotc46Uy9wYEpdGrtDG\/thWay3fmMsjpWR0U25xFIrxTrfCGBblYpD7juukWWml2E9rtE2rHgUxbymxXjEw7xrMwcGrhOGyqwoBqJy1JVwIDAQABo4ICLTCCAikwZAYIKwYBBQUHAQEEWDBWMFQGCCsGAQUFBzABhkhodHRwOi8vY29yZW9jc3AuZ2xvYmFsLnN0YW5kYXJkY2hhcnRlcmVkLmNvbS9lamJjYS9wdWJsaWN3ZWIvc3RhdHVzL29jc3AwHQYDVR0OBBYEFIinW4BNDeVEFcuLf8YjZjtySoW9MAwGA1UdEwEB\/wQCMAAwHwYDVR0jBBgwFoAUfNZMoZi33nKrcmVU3TFVQnuEi\/4wggFCBgNVHR8EggE5MIIBNTCCATGggcKggb+GgbxodHRwOi8vY29yZWNybC5nbG9iYWwuc3RhbmRhcmRjaGFydGVyZWQuY29tL2VqYmNhL3B1YmxpY3dlYi93ZWJkaXN0L2NlcnRkaXN0P2NtZD1jcmwmaXNzdWVyPUNOPVN0YW5kYXJkJTIwQ2hhcnRlcmVkJTIwQmFuayUyMFNpZ25pbmclMjBDQSUyMEcxJTIwLSUyMFNIQTIsTz1TdGFuZGFyZCUyMENoYXJ0ZXJlZCUyMEJhbmssQz1HQqJqpGgwZjE1MDMGA1UEAwwsU3RhbmRhcmQgQ2hhcnRlcmVkIEJhbmsgU2lnbmluZyBDQSBHMSAtIFNIQTIxIDAeBgNVBAoMF1N0YW5kYXJkIENoYXJ0ZXJlZCBCYW5rMQswCQYDVQQGEwJHQjAOBgNVHQ8BAf8EBAMCBsAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMA0GCSqGSIb3DQEBCwUAA4ICAQBtsoRlDHuOTDChcWdfdVUtRgP0U0ijDSeJi8vULN1rgYnqqJc4PdJno50aiu9MGlxY02O7HW7ZVD6QEG\/pqHmZ0sbWpb\/fumMgZSjP65IcGuS53zgcNtLYnyXyEv+v5T\/CK3bk4Li6tUW3ScJPUwVWwP1E0\/u6aBSb5k\/h4lTwS1o88ybS5pJOg6XutXByp991QQrrs7tp7fKNynjNZbFuG3J1e09X+zTfJOpjaDUofQTkt8IyMRI6Cs4wI1eZA+dAIL8B0n8ze1mRl1FOJqgdZrAQjoqZkCTnc0Il5VY\/dUXxGVg6D9e5pfck3FWT107K9\/5EZoxytpqYXFCjMXi5hx4YjK17OUgm82mZhvqkNdzF8Yq2vFuB3LPfyelESq99xFLykvinrVm1NtZKeDTT1Jq\/VvZt6stO\/tovq1RfJJcznpYcwOzxlnhGR6E+hxuBx7aDJzGf0JaoRxQILH1B2XV9WDI3HPYQsP7XtriX+QUJ\/aly28QkV48RmaGYCsly43YZu1MKudSsw+dhnbZzRsg\/aes3dzGW2x137bQPtux7k2LCSpsTXgedhOys28YoGlsoe8kUv0myAU4Stt+I3mrwO3BKUn+tJggvlDiiiyT1tg2HiklyU\/2FxQkZRMeB0eRrXTpg3l9x2mpF+dDFxOMKszxwD2kgoEZgo6o58A==", line too long (2378 > 90) http://gerrit.cloudera.org:8080/#/c/21382/4/be/src/util/jwt-util-test.cc@386 PS4, Line 386: "MIIE2jCCAsICAQEwDQYJKoZIhvcNAQELBQAwMzELMAkGA1UEBhMCVVMxEDAOBgNVBAoMB0pXVC1DUFAxEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0yMzEyMjIxMzIzNTdaFw0zMzEyMTkxMzIzNTdaMDMxCzAJBgNVBAYTAlVTMRAwDgYDVQQKDAdKV1QtQ1BQMRIwEAYDVQQDDAlsb2NhbGhvc3QwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDl0gyL9KCpoXsJlvBaBUsJTAgvFhbgsBjpxT6m2xv0fgtGDBwgaiPvuMnClTU/kYkKb8c1GTkMedKp9YcM57HWTk9yqGTy6QBnMMxbAJYNwWQ4Dbr4qKSC6C3KzYws/Bqyv8OC9NAOyqJbtdp4iObRjyaet+PLTXywuu02xtyRg3B+1aAONgUVDyS5u57NSD4rEZ+rw30Ne1doSClWmMDqEd72y8cjx3eAqn0HcAxSQ6MNMmNk7/M8FQD3DTM1Ef0G5oHyJIw7WmY+gxuD8386r/CkswINzadMwObPlTSdAN8BRzedtrqgb+D/K4pi2zhCiuIVujFX6M/hsGvj7g2M9E9MR8iEuHWCY9frQKIR+JTH3D1snoJp60qKoa51qBznsEr9RP2utGniPCq3+JY+ZX0JK8vl5tiSZpy6N0yRbRmY3XLdA5fKRzhcsB3eUrmTtr9ywjZX7Ll6QMvUyicubGTojhqJFQbvuvvops9PoCMXFE3x6cJ2QhPoi8+BvUdYisrtjDFe+YgrgQvPMa/CpOpDJJDEs2SVRcauCZOUdqLCwZylNuW0CgIjWP8l99P7l1zGeT8VJPhmABYyPM+RtNYDamAlUOCqRqgz/gPjEeMeulQTvH1lAqATAAX1oftlq6o4VoqROs2M3eAXqPhvsLBeTmCob+5ca887MkcP6wIDAQABMA0GCSqGSIb3DQEBCwUAA4ICAQBW2kREK4hlzxCDqykxrwfbQpiPwrbFmn+3RDJla+pI4L3wrvYT1nU96guFIU3zKnbMzqwPMRUCUjadr2jKxAmMWxCd/ThHQB+ne5xTvx7/6RVQfGjyMCG/SZtSH8/aO7ILNRtPT+SL5ZZwezaqv6gD89tSXB/w/0pYXy70wDuU17KCrTsKSISWGJ1cKi5l2R/m/ZaGjcV8U8NcFepF2bX3u/i0zhaqOqjiwrSEt7fWGDLabPs6n7GtfibZROEDZ/h0JrDINC+6mSfTOYAMJvGjeHA3H/NvzqR+CJgpXGCqElqVuBF0HdxPmwRRBoZC/BLIEcz0VHmB4rcpfaV47TZT+J+04fHYp4Y1S0u112CDrDe+61cDrnbDHC7aGX0G93pYSBKAB1e3LLc9rXQgf2F0pRtFB3rgZA9MtJ+TL7DUvY4VXJNq3v7UolIdldYRdk21YqAS2Hp0fivvFoEk2P/WbwDEErxR0FkZ/JQoI9FMJ9AvDxa4MsFFtlQVInfD2HUu+nhnuEAA8R6L+F2XqhfLY/H7H31iFBK6UCuqptED71VwWHqfBsAPRhLXAqGco7Ln2dzioyj0QdwJqQQIqigltSYtXxfIMLW0BekQ5yln7QTxnZlobkPHUW9s3NK+OMLuKCzVREzjic/aioQP3cRBMXkG2deMwrk3aX8yJuz4gA==" line too long (1682 > 90) http://gerrit.cloudera.org:8080/#/c/21382/4/be/src/util/jwt-util-test.cc@388 PS4, Line 388: "n":
[Impala-ASF-CR] WIP
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21382 ) Change subject: WIP .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/16074/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21382 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I70be6f9f54190544aa005b2644e2ed8db6f6bb74 Gerrit-Change-Number: 21382 Gerrit-PatchSet: 3 Gerrit-Owner: gaurav singh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 01 May 2024 16:27:03 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21382 ) Change subject: WIP .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/16073/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21382 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I70be6f9f54190544aa005b2644e2ed8db6f6bb74 Gerrit-Change-Number: 21382 Gerrit-PatchSet: 2 Gerrit-Owner: gaurav singh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 01 May 2024 16:14:23 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21382 ) Change subject: WIP .. Patch Set 3: (8 comments) http://gerrit.cloudera.org:8080/#/c/21382/3/be/src/util/jwt-util-test.cc File be/src/util/jwt-util-test.cc: http://gerrit.cloudera.org:8080/#/c/21382/3/be/src/util/jwt-util-test.cc@384 PS3, Line 384: "MIIG1TCCBL2gAwIBAgIIFvMVGp6t\/cMwDQYJKoZIhvcNAQELBQAwZjELMAkGA1UEBhMCR0IxIDAeBgNVBAoMF1N0YW5kYXJkIENoYXJ0ZXJlZCBCYW5rMTUwMwYDVQQDDCxTdGFuZGFyZCBDaGFydGVyZWQgQmFuayBTaWduaW5nIENBIEcxIC0gU0hBMjAeFw0xODEwMTAxMTI2MzVaFw0yMjEwMTAxMTI2MzVaMIG9MQswCQYDVQQGEwJTRzESMBAGA1UECAwJU2luZ2Fwb3JlMRIwEAYDVQQHDAlTaW5nYXBvcmUxIDAeBgNVBAoMF1N0YW5kYXJkIENoYXJ0ZXJlZCBCYW5rMRwwGgYDVQQLDBNGb3VuZGF0aW9uIFNlcnZpY2VzMSgwJgYDVQQDDB9pbnRlcm5hbC1nYXRld2F5LWp3dC5hcGkuc2MubmV0MRwwGgYJKoZIhvcNAQkBFg1BUElQU1NAc2MuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArVWBoIi3IJ4nOWXu7\/SDxczqMou1B+c4c2FdQrOXrK31HxAaz4WEtma9BLXFdFHJ5mCCPIvdUcVxxnCynqhMOkZ\/a7acQbUD9cDzI8isMB9JL7VooDw0CctxHxffjqQQVIEhC2Q7zsM1pQayR7cl+pbBlvHIoRxq2n1B0fFvfoiosjf4kDiCpgHdM+v5Hw9aVYmUbroHxmQWqhB0iRTJQPPLZqqQVC50A1Q\/96gkwoODyotc46Uy9wYEpdGrtDG\/thWay3fmMsjpWR0U25xFIrxTrfCGBblYpD7juukWWml2E9rtE2rHgUxbymxXjEw7xrMwcGrhOGyqwoBqJy1JVwIDAQABo4ICLTCCAikwZAYIKwYBBQUHAQEEWDBWMFQGCCsGAQUFBzABhkhodHRwOi8vY29yZW9jc3AuZ2xvYmFsLnN0YW5kYXJkY2hhcnRlcmVkLmNvbS9lamJjYS9wdWJsaWN3ZWIvc3RhdHVzL29jc3AwHQYDVR0OBBYEFIinW4BNDeVEFcuLf8YjZjtySoW9MAwGA1UdEwEB\/wQCMAAwHwYDVR0jBBgwFoAUfNZMoZi33nKrcmVU3TFVQnuEi\/4wggFCBgNVHR8EggE5MIIBNTCCATGggcKggb+GgbxodHRwOi8vY29yZWNybC5nbG9iYWwuc3RhbmRhcmRjaGFydGVyZWQuY29tL2VqYmNhL3B1YmxpY3dlYi93ZWJkaXN0L2NlcnRkaXN0P2NtZD1jcmwmaXNzdWVyPUNOPVN0YW5kYXJkJTIwQ2hhcnRlcmVkJTIwQmFuayUyMFNpZ25pbmclMjBDQSUyMEcxJTIwLSUyMFNIQTIsTz1TdGFuZGFyZCUyMENoYXJ0ZXJlZCUyMEJhbmssQz1HQqJqpGgwZjE1MDMGA1UEAwwsU3RhbmRhcmQgQ2hhcnRlcmVkIEJhbmsgU2lnbmluZyBDQSBHMSAtIFNIQTIxIDAeBgNVBAoMF1N0YW5kYXJkIENoYXJ0ZXJlZCBCYW5rMQswCQYDVQQGEwJHQjAOBgNVHQ8BAf8EBAMCBsAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMA0GCSqGSIb3DQEBCwUAA4ICAQBtsoRlDHuOTDChcWdfdVUtRgP0U0ijDSeJi8vULN1rgYnqqJc4PdJno50aiu9MGlxY02O7HW7ZVD6QEG\/pqHmZ0sbWpb\/fumMgZSjP65IcGuS53zgcNtLYnyXyEv+v5T\/CK3bk4Li6tUW3ScJPUwVWwP1E0\/u6aBSb5k\/h4lTwS1o88ybS5pJOg6XutXByp991QQrrs7tp7fKNynjNZbFuG3J1e09X+zTfJOpjaDUofQTkt8IyMRI6Cs4wI1eZA+dAIL8B0n8ze1mRl1FOJqgdZrAQjoqZkCTnc0Il5VY\/dUXxGVg6D9e5pfck3FWT107K9\/5EZoxytpqYXFCjMXi5hx4YjK17OUgm82mZhvqkNdzF8Yq2vFuB3LPfyelESq99xFLykvinrVm1NtZKeDTT1Jq\/VvZt6stO\/tovq1RfJJcznpYcwOzxlnhGR6E+hxuBx7aDJzGf0JaoRxQILH1B2XV9WDI3HPYQsP7XtriX+QUJ\/aly28QkV48RmaGYCsly43YZu1MKudSsw+dhnbZzRsg\/aes3dzGW2x137bQPtux7k2LCSpsTXgedhOys28YoGlsoe8kUv0myAU4Stt+I3mrwO3BKUn+tJggvlDiiiyT1tg2HiklyU\/2FxQkZRMeB0eRrXTpg3l9x2mpF+dDFxOMKszxwD2kgoEZgo6o58A==", line too long (2378 > 90) http://gerrit.cloudera.org:8080/#/c/21382/3/be/src/util/jwt-util-test.cc@386 PS3, Line 386: "MIIE2jCCAsICAQEwDQYJKoZIhvcNAQELBQAwMzELMAkGA1UEBhMCVVMxEDAOBgNVBAoMB0pXVC1DUFAxEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0yMzEyMjIxMzIzNTdaFw0zMzEyMTkxMzIzNTdaMDMxCzAJBgNVBAYTAlVTMRAwDgYDVQQKDAdKV1QtQ1BQMRIwEAYDVQQDDAlsb2NhbGhvc3QwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDl0gyL9KCpoXsJlvBaBUsJTAgvFhbgsBjpxT6m2xv0fgtGDBwgaiPvuMnClTU/kYkKb8c1GTkMedKp9YcM57HWTk9yqGTy6QBnMMxbAJYNwWQ4Dbr4qKSC6C3KzYws/Bqyv8OC9NAOyqJbtdp4iObRjyaet+PLTXywuu02xtyRg3B+1aAONgUVDyS5u57NSD4rEZ+rw30Ne1doSClWmMDqEd72y8cjx3eAqn0HcAxSQ6MNMmNk7/M8FQD3DTM1Ef0G5oHyJIw7WmY+gxuD8386r/CkswINzadMwObPlTSdAN8BRzedtrqgb+D/K4pi2zhCiuIVujFX6M/hsGvj7g2M9E9MR8iEuHWCY9frQKIR+JTH3D1snoJp60qKoa51qBznsEr9RP2utGniPCq3+JY+ZX0JK8vl5tiSZpy6N0yRbRmY3XLdA5fKRzhcsB3eUrmTtr9ywjZX7Ll6QMvUyicubGTojhqJFQbvuvvops9PoCMXFE3x6cJ2QhPoi8+BvUdYisrtjDFe+YgrgQvPMa/CpOpDJJDEs2SVRcauCZOUdqLCwZylNuW0CgIjWP8l99P7l1zGeT8VJPhmABYyPM+RtNYDamAlUOCqRqgz/gPjEeMeulQTvH1lAqATAAX1oftlq6o4VoqROs2M3eAXqPhvsLBeTmCob+5ca887MkcP6wIDAQABMA0GCSqGSIb3DQEBCwUAA4ICAQBW2kREK4hlzxCDqykxrwfbQpiPwrbFmn+3RDJla+pI4L3wrvYT1nU96guFIU3zKnbMzqwPMRUCUjadr2jKxAmMWxCd/ThHQB+ne5xTvx7/6RVQfGjyMCG/SZtSH8/aO7ILNRtPT+SL5ZZwezaqv6gD89tSXB/w/0pYXy70wDuU17KCrTsKSISWGJ1cKi5l2R/m/ZaGjcV8U8NcFepF2bX3u/i0zhaqOqjiwrSEt7fWGDLabPs6n7GtfibZROEDZ/h0JrDINC+6mSfTOYAMJvGjeHA3H/NvzqR+CJgpXGCqElqVuBF0HdxPmwRRBoZC/BLIEcz0VHmB4rcpfaV47TZT+J+04fHYp4Y1S0u112CDrDe+61cDrnbDHC7aGX0G93pYSBKAB1e3LLc9rXQgf2F0pRtFB3rgZA9MtJ+TL7DUvY4VXJNq3v7UolIdldYRdk21YqAS2Hp0fivvFoEk2P/WbwDEErxR0FkZ/JQoI9FMJ9AvDxa4MsFFtlQVInfD2HUu+nhnuEAA8R6L+F2XqhfLY/H7H31iFBK6UCuqptED71VwWHqfBsAPRhLXAqGco7Ln2dzioyj0QdwJqQQIqigltSYtXxfIMLW0BekQ5yln7QTxnZlobkPHUW9s3NK+OMLuKCzVREzjic/aioQP3cRBMXkG2deMwrk3aX8yJuz4gA==", line too long (1673 > 90) http://gerrit.cloudera.org:8080/#/c/21382/3/be/src/util/jwt-util-test.cc@389 PS3, Line 389: "n":
[Impala-ASF-CR] WIP
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21382 to look at the new patch set (#3). Change subject: WIP .. WIP IMPALA-12559: Support x5c Parameter in JSON Web Keys This enables the jwt verification using the x5c certificates in the jwks key. The x5c claim is part of the jwks either as a string or an array. This patch only supports one x5c certificate per public key. Testing: * added unit test case VerifyJwtTokenWithx5cCertificate to verify jwt with x5c certificate. Change-Id: I70be6f9f54190544aa005b2644e2ed8db6f6bb74 --- M be/src/util/jwt-util-test.cc M be/src/util/jwt-util.cc 2 files changed, 200 insertions(+), 19 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/21382/3 -- To view, visit http://gerrit.cloudera.org:8080/21382 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I70be6f9f54190544aa005b2644e2ed8db6f6bb74 Gerrit-Change-Number: 21382 Gerrit-PatchSet: 3 Gerrit-Owner: gaurav singh Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] WIP
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21382 ) Change subject: WIP .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/16072/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21382 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I70be6f9f54190544aa005b2644e2ed8db6f6bb74 Gerrit-Change-Number: 21382 Gerrit-PatchSet: 1 Gerrit-Owner: gaurav singh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 01 May 2024 15:56:07 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21382 ) Change subject: WIP .. Patch Set 2: (15 comments) http://gerrit.cloudera.org:8080/#/c/21382/2/be/src/util/jwt-util-test.cc File be/src/util/jwt-util-test.cc: http://gerrit.cloudera.org:8080/#/c/21382/2/be/src/util/jwt-util-test.cc@384 PS2, Line 384: "MIIG1TCCBL2gAwIBAgIIFvMVGp6t\/cMwDQYJKoZIhvcNAQELBQAwZjELMAkGA1UEBhMCR0IxIDAeBgNVBAoMF1N0YW5kYXJkIENoYXJ0ZXJlZCBCYW5rMTUwMwYDVQQDDCxTdGFuZGFyZCBDaGFydGVyZWQgQmFuayBTaWduaW5nIENBIEcxIC0gU0hBMjAeFw0xODEwMTAxMTI2MzVaFw0yMjEwMTAxMTI2MzVaMIG9MQswCQYDVQQGEwJTRzESMBAGA1UECAwJU2luZ2Fwb3JlMRIwEAYDVQQHDAlTaW5nYXBvcmUxIDAeBgNVBAoMF1N0YW5kYXJkIENoYXJ0ZXJlZCBCYW5rMRwwGgYDVQQLDBNGb3VuZGF0aW9uIFNlcnZpY2VzMSgwJgYDVQQDDB9pbnRlcm5hbC1nYXRld2F5LWp3dC5hcGkuc2MubmV0MRwwGgYJKoZIhvcNAQkBFg1BUElQU1NAc2MuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArVWBoIi3IJ4nOWXu7\/SDxczqMou1B+c4c2FdQrOXrK31HxAaz4WEtma9BLXFdFHJ5mCCPIvdUcVxxnCynqhMOkZ\/a7acQbUD9cDzI8isMB9JL7VooDw0CctxHxffjqQQVIEhC2Q7zsM1pQayR7cl+pbBlvHIoRxq2n1B0fFvfoiosjf4kDiCpgHdM+v5Hw9aVYmUbroHxmQWqhB0iRTJQPPLZqqQVC50A1Q\/96gkwoODyotc46Uy9wYEpdGrtDG\/thWay3fmMsjpWR0U25xFIrxTrfCGBblYpD7juukWWml2E9rtE2rHgUxbymxXjEw7xrMwcGrhOGyqwoBqJy1JVwIDAQABo4ICLTCCAikwZAYIKwYBBQUHAQEEWDBWMFQGCCsGAQUFBzABhkhodHRwOi8vY29yZW9jc3AuZ2xvYmFsLnN0YW5kYXJkY2hhcnRlcmVkLmNvbS9lamJjYS9wdWJsaWN3ZWIvc3RhdHVzL29jc3AwHQYDVR0OBBYEFIinW4BNDeVEFcuLf8YjZjtySoW9MAwGA1UdEwEB\/wQCMAAwHwYDVR0jBBgwFoAUfNZMoZi33nKrcmVU3TFVQnuEi\/4wggFCBgNVHR8EggE5MIIBNTCCATGggcKggb+GgbxodHRwOi8vY29yZWNybC5nbG9iYWwuc3RhbmRhcmRjaGFydGVyZWQuY29tL2VqYmNhL3B1YmxpY3dlYi93ZWJkaXN0L2NlcnRkaXN0P2NtZD1jcmwmaXNzdWVyPUNOPVN0YW5kYXJkJTIwQ2hhcnRlcmVkJTIwQmFuayUyMFNpZ25pbmclMjBDQSUyMEcxJTIwLSUyMFNIQTIsTz1TdGFuZGFyZCUyMENoYXJ0ZXJlZCUyMEJhbmssQz1HQqJqpGgwZjE1MDMGA1UEAwwsU3RhbmRhcmQgQ2hhcnRlcmVkIEJhbmsgU2lnbmluZyBDQSBHMSAtIFNIQTIxIDAeBgNVBAoMF1N0YW5kYXJkIENoYXJ0ZXJlZCBCYW5rMQswCQYDVQQGEwJHQjAOBgNVHQ8BAf8EBAMCBsAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMA0GCSqGSIb3DQEBCwUAA4ICAQBtsoRlDHuOTDChcWdfdVUtRgP0U0ijDSeJi8vULN1rgYnqqJc4PdJno50aiu9MGlxY02O7HW7ZVD6QEG\/pqHmZ0sbWpb\/fumMgZSjP65IcGuS53zgcNtLYnyXyEv+v5T\/CK3bk4Li6tUW3ScJPUwVWwP1E0\/u6aBSb5k\/h4lTwS1o88ybS5pJOg6XutXByp991QQrrs7tp7fKNynjNZbFuG3J1e09X+zTfJOpjaDUofQTkt8IyMRI6Cs4wI1eZA+dAIL8B0n8ze1mRl1FOJqgdZrAQjoqZkCTnc0Il5VY\/dUXxGVg6D9e5pfck3FWT107K9\/5EZoxytpqYXFCjMXi5hx4YjK17OUgm82mZhvqkNdzF8Yq2vFuB3LPfyelESq99xFLykvinrVm1NtZKeDTT1Jq\/VvZt6stO\/tovq1RfJJcznpYcwOzxlnhGR6E+hxuBx7aDJzGf0JaoRxQILH1B2XV9WDI3HPYQsP7XtriX+QUJ\/aly28QkV48RmaGYCsly43YZu1MKudSsw+dhnbZzRsg\/aes3dzGW2x137bQPtux7k2LCSpsTXgedhOys28YoGlsoe8kUv0myAU4Stt+I3mrwO3BKUn+tJggvlDiiiyT1tg2HiklyU\/2FxQkZRMeB0eRrXTpg3l9x2mpF+dDFxOMKszxwD2kgoEZgo6o58A==", line too long (2378 > 90) http://gerrit.cloudera.org:8080/#/c/21382/2/be/src/util/jwt-util-test.cc@386 PS2, Line 386: "MIIE2jCCAsICAQEwDQYJKoZIhvcNAQELBQAwMzELMAkGA1UEBhMCVVMxEDAOBgNVBAoMB0pXVC1DUFAxEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0yMzEyMjIxMzIzNTdaFw0zMzEyMTkxMzIzNTdaMDMxCzAJBgNVBAYTAlVTMRAwDgYDVQQKDAdKV1QtQ1BQMRIwEAYDVQQDDAlsb2NhbGhvc3QwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDl0gyL9KCpoXsJlvBaBUsJTAgvFhbgsBjpxT6m2xv0fgtGDBwgaiPvuMnClTU/kYkKb8c1GTkMedKp9YcM57HWTk9yqGTy6QBnMMxbAJYNwWQ4Dbr4qKSC6C3KzYws/Bqyv8OC9NAOyqJbtdp4iObRjyaet+PLTXywuu02xtyRg3B+1aAONgUVDyS5u57NSD4rEZ+rw30Ne1doSClWmMDqEd72y8cjx3eAqn0HcAxSQ6MNMmNk7/M8FQD3DTM1Ef0G5oHyJIw7WmY+gxuD8386r/CkswINzadMwObPlTSdAN8BRzedtrqgb+D/K4pi2zhCiuIVujFX6M/hsGvj7g2M9E9MR8iEuHWCY9frQKIR+JTH3D1snoJp60qKoa51qBznsEr9RP2utGniPCq3+JY+ZX0JK8vl5tiSZpy6N0yRbRmY3XLdA5fKRzhcsB3eUrmTtr9ywjZX7Ll6QMvUyicubGTojhqJFQbvuvvops9PoCMXFE3x6cJ2QhPoi8+BvUdYisrtjDFe+YgrgQvPMa/CpOpDJJDEs2SVRcauCZOUdqLCwZylNuW0CgIjWP8l99P7l1zGeT8VJPhmABYyPM+RtNYDamAlUOCqRqgz/gPjEeMeulQTvH1lAqATAAX1oftlq6o4VoqROs2M3eAXqPhvsLBeTmCob+5ca887MkcP6wIDAQABMA0GCSqGSIb3DQEBCwUAA4ICAQBW2kREK4hlzxCDqykxrwfbQpiPwrbFmn+3RDJla+pI4L3wrvYT1nU96guFIU3zKnbMzqwPMRUCUjadr2jKxAmMWxCd/ThHQB+ne5xTvx7/6RVQfGjyMCG/SZtSH8/aO7ILNRtPT+SL5ZZwezaqv6gD89tSXB/w/0pYXy70wDuU17KCrTsKSISWGJ1cKi5l2R/m/ZaGjcV8U8NcFepF2bX3u/i0zhaqOqjiwrSEt7fWGDLabPs6n7GtfibZROEDZ/h0JrDINC+6mSfTOYAMJvGjeHA3H/NvzqR+CJgpXGCqElqVuBF0HdxPmwRRBoZC/BLIEcz0VHmB4rcpfaV47TZT+J+04fHYp4Y1S0u112CDrDe+61cDrnbDHC7aGX0G93pYSBKAB1e3LLc9rXQgf2F0pRtFB3rgZA9MtJ+TL7DUvY4VXJNq3v7UolIdldYRdk21YqAS2Hp0fivvFoEk2P/WbwDEErxR0FkZ/JQoI9FMJ9AvDxa4MsFFtlQVInfD2HUu+nhnuEAA8R6L+F2XqhfLY/H7H31iFBK6UCuqptED71VwWHqfBsAPRhLXAqGco7Ln2dzioyj0QdwJqQQIqigltSYtXxfIMLW0BekQ5yln7QTxnZlobkPHUW9s3NK+OMLuKCzVREzjic/aioQP3cRBMXkG2deMwrk3aX8yJuz4gA==", line too long (1673 > 90) http://gerrit.cloudera.org:8080/#/c/21382/2/be/src/util/jwt-util-test.cc@389 PS2, Line 389: "n":
[Impala-ASF-CR] WIP
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21382 to look at the new patch set (#2). Change subject: WIP .. WIP IMPALA-12559: Support x5c Parameter in JSON Web Keys This enables the jwt verification using the x5c certificates in the jwks key. The x5c claim is part of the jwks either as a string or an array. This patch only supports one x5c certificate per public key. Testing: * added unit test case VerifyJwtTokenWithx5cCertificate to verify jwt with x5c certificate. Change-Id: I70be6f9f54190544aa005b2644e2ed8db6f6bb74 --- M be/src/util/jwt-util-test.cc M be/src/util/jwt-util.cc 2 files changed, 197 insertions(+), 19 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/21382/2 -- To view, visit http://gerrit.cloudera.org:8080/21382 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I70be6f9f54190544aa005b2644e2ed8db6f6bb74 Gerrit-Change-Number: 21382 Gerrit-PatchSet: 2 Gerrit-Owner: gaurav singh Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
cclive1...@gmail.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 ) Change subject: [WIP]Hierarchical metastore event processing .. Patch Set 12: (1 comment) http://gerrit.cloudera.org:8080/#/c/21031/12/fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java File fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java: http://gerrit.cloudera.org:8080/#/c/21031/12/fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java@213 PS12, Line 213: eventProcessor_.getStatus()); of we can just mark this log as debug is debug is enabled. -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 12 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 26 Apr 2024 07:50:26 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
cclive1...@gmail.com has posted comments on this change. (
http://gerrit.cloudera.org:8080/21031 )
Change subject: [WIP]Hierarchical metastore event processing
..
Patch Set 12:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/21031/12/fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java
File fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java:
http://gerrit.cloudera.org:8080/#/c/21031/12/fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java@212
PS12, Line 212: LOG.info("[{}] Clear event processor since status is {}",
executorName_,
I think we can remove this log or using some rate log method to print log ,I
tested this code ,and I got many unused logs, which will use much storage (it
printed 5 log files (200M/per file) just in 5min), and most of them a useless
and invalid.
--
To view, visit http://gerrit.cloudera.org:8080/21031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6
Gerrit-Change-Number: 21031
Gerrit-PatchSet: 12
Gerrit-Owner: Anonymous Coward
Gerrit-Reviewer: Anonymous Coward
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Fri, 26 Apr 2024 07:46:03 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 ) Change subject: [WIP]Hierarchical metastore event processing .. Patch Set 12: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/16008/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 12 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 24 Apr 2024 18:15:54 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21031 to look at the new patch set (#12). Change subject: [WIP]Hierarchical metastore event processing .. [WIP]Hierarchical metastore event processing Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M be/src/util/event-metrics.cc M be/src/util/event-metrics.h M common/thrift/BackendGflags.thrift M common/thrift/JniCatalog.thrift M common/thrift/metrics.json M fe/src/compat-apache-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/TableWriteId.java A fe/src/main/java/org/apache/impala/catalog/events/DBBarrierEvent.java A fe/src/main/java/org/apache/impala/catalog/events/DBEventExecutor.java M fe/src/main/java/org/apache/impala/catalog/events/ExternalEventsProcessor.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/main/java/org/apache/impala/catalog/events/NoOpEventProcessor.java A fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/DebugUtils.java M fe/src/test/java/org/apache/impala/catalog/CatalogTableWriteIdTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java A tests/custom_cluster/test_event_processing_perf.py M tests/util/event_processor_utils.py 25 files changed, 1,627 insertions(+), 73 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/21031/12 -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 12 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 ) Change subject: [WIP]Hierarchical metastore event processing .. Patch Set 12: (1 comment) http://gerrit.cloudera.org:8080/#/c/21031/12/tests/custom_cluster/test_event_processing_perf.py File tests/custom_cluster/test_event_processing_perf.py: http://gerrit.cloudera.org:8080/#/c/21031/12/tests/custom_cluster/test_event_processing_perf.py@217 PS12, Line 217: ( flake8: E225 missing whitespace around operator -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 12 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 24 Apr 2024 17:53:07 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 ) Change subject: [WIP]Hierarchical metastore event processing .. Patch Set 11: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/16007/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 11 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 24 Apr 2024 16:15:27 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 ) Change subject: [WIP]Hierarchical metastore event processing .. Patch Set 11: (1 comment) http://gerrit.cloudera.org:8080/#/c/21031/11/tests/custom_cluster/test_event_processing_perf.py File tests/custom_cluster/test_event_processing_perf.py: http://gerrit.cloudera.org:8080/#/c/21031/11/tests/custom_cluster/test_event_processing_perf.py@217 PS11, Line 217: ( flake8: E225 missing whitespace around operator -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 11 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 24 Apr 2024 15:51:27 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21031 to look at the new patch set (#11). Change subject: [WIP]Hierarchical metastore event processing .. [WIP]Hierarchical metastore event processing Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M be/src/util/event-metrics.cc M be/src/util/event-metrics.h M common/thrift/BackendGflags.thrift M common/thrift/JniCatalog.thrift M common/thrift/metrics.json M fe/src/compat-apache-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/TableWriteId.java A fe/src/main/java/org/apache/impala/catalog/events/DBBarrierEvent.java A fe/src/main/java/org/apache/impala/catalog/events/DBEventExecutor.java M fe/src/main/java/org/apache/impala/catalog/events/ExternalEventsProcessor.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/main/java/org/apache/impala/catalog/events/NoOpEventProcessor.java A fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/DebugUtils.java M fe/src/test/java/org/apache/impala/catalog/CatalogTableWriteIdTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java A tests/custom_cluster/test_event_processing_perf.py M tests/util/event_processor_utils.py 25 files changed, 1,647 insertions(+), 73 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/21031/11 -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 11 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 ) Change subject: [WIP]Hierarchical metastore event processing .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/16006/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 10 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 24 Apr 2024 15:35:52 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 ) Change subject: [WIP]Hierarchical metastore event processing .. Patch Set 10: (6 comments) http://gerrit.cloudera.org:8080/#/c/21031/10/tests/custom_cluster/test_event_processing_perf.py File tests/custom_cluster/test_event_processing_perf.py: http://gerrit.cloudera.org:8080/#/c/21031/10/tests/custom_cluster/test_event_processing_perf.py@20 PS10, Line 20: from os import getenv flake8: F401 'os.getenv' imported but unused http://gerrit.cloudera.org:8080/#/c/21031/10/tests/custom_cluster/test_event_processing_perf.py@25 PS10, Line 25: from tests.common.skip import SkipIf, SkipIfFS flake8: F401 'tests.common.skip.SkipIf' imported but unused http://gerrit.cloudera.org:8080/#/c/21031/10/tests/custom_cluster/test_event_processing_perf.py@31 PS10, Line 31: @SkipIfFS.hive flake8: E302 expected 2 blank lines, found 1 http://gerrit.cloudera.org:8080/#/c/21031/10/tests/custom_cluster/test_event_processing_perf.py@217 PS10, Line 217: ( flake8: E225 missing whitespace around operator http://gerrit.cloudera.org:8080/#/c/21031/10/tests/util/event_processor_utils.py File tests/util/event_processor_utils.py: http://gerrit.cloudera.org:8080/#/c/21031/10/tests/util/event_processor_utils.py@41 PS10, Line 41: a flake8: E501 line too long (105 > 90 characters) http://gerrit.cloudera.org:8080/#/c/21031/10/tests/util/event_processor_utils.py@60 PS10, Line 60: = flake8: E501 line too long (95 > 90 characters) -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 10 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 24 Apr 2024 15:12:22 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21031 to look at the new patch set (#10). Change subject: [WIP]Hierarchical metastore event processing .. [WIP]Hierarchical metastore event processing Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M be/src/util/event-metrics.cc M be/src/util/event-metrics.h M common/thrift/BackendGflags.thrift M common/thrift/JniCatalog.thrift M common/thrift/metrics.json M fe/src/compat-apache-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/TableWriteId.java A fe/src/main/java/org/apache/impala/catalog/events/DBBarrierEvent.java A fe/src/main/java/org/apache/impala/catalog/events/DBEventExecutor.java M fe/src/main/java/org/apache/impala/catalog/events/ExternalEventsProcessor.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/main/java/org/apache/impala/catalog/events/NoOpEventProcessor.java A fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/DebugUtils.java M fe/src/test/java/org/apache/impala/catalog/CatalogTableWriteIdTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java A tests/custom_cluster/test_event_processing_perf.py M tests/util/event_processor_utils.py 25 files changed, 1,645 insertions(+), 73 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/21031/10 -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 10 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] WIP: IMPALA-12046: Add profile counter for scan range queueing time on disk queues
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21309 ) Change subject: WIP: IMPALA-12046: Add profile counter for scan range queueing time on disk queues .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15910/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21309 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia0cb24db36cd89933c150b88e166e68a3abc7a60 Gerrit-Change-Number: 21309 Gerrit-PatchSet: 1 Gerrit-Owner: Yida Wu Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Apr 2024 17:42:15 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-12046: Add profile counter for scan range queueing time on disk queues
Yida Wu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21309 Change subject: WIP: IMPALA-12046: Add profile counter for scan range queueing time on disk queues .. WIP: IMPALA-12046: Add profile counter for scan range queueing time on disk queues This patch introduces a new timer counter in the query profile to track the time spent by scan ranges in the disk io queues. The solution is to add a timer to each request range and starting and stopping the timer when the request range is in the disk io queue. Each request context will have a queueing timer counter, and every timer associated with the range belonging to the request context will aggregate the time count to this counter. Also adds Reset() function to existing timer to enable fast resetting of the timer. Since ranges are now reusable, it is more efficient to reset the timer instead of reallocating it every time the range is reused. Tests: Manuall tests and it shows the aggregated queueing time in the profile. TODO: Test the performance impact and add tests. Change-Id: Ia0cb24db36cd89933c150b88e166e68a3abc7a60 --- M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-base.h M be/src/runtime/io/disk-io-mgr.cc M be/src/runtime/io/request-context.cc M be/src/runtime/io/request-context.h M be/src/runtime/io/request-ranges.h M be/src/util/runtime-profile-counters.h M be/src/util/stopwatch.h 8 files changed, 90 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/09/21309/1 -- To view, visit http://gerrit.cloudera.org:8080/21309 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia0cb24db36cd89933c150b88e166e68a3abc7a60 Gerrit-Change-Number: 21309 Gerrit-PatchSet: 1 Gerrit-Owner: Yida Wu
[Impala-ASF-CR] WIP IMPALA-12910: Support running TPCH/TPCDS queries for JDBC tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21304 ) Change subject: WIP IMPALA-12910: Support running TPCH/TPCDS queries for JDBC tables .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15904/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I44e8c1bb020e90559c7f22483a7ab7a151b8f48a Gerrit-Change-Number: 21304 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Apr 2024 02:59:34 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-12910: Support running TPCH/TPCDS queries for JDBC tables
Wenzhe Zhou has uploaded a new patch set (#2). (
http://gerrit.cloudera.org:8080/21304 )
Change subject: WIP IMPALA-12910: Support running TPCH/TPCDS queries for JDBC
tables
..
WIP IMPALA-12910: Support running TPCH/TPCDS queries for JDBC tables
This patch adds script to create external JDBC tables for the dataset
of TPCH and TPCDS, and adds unit-tests to run TPCH and TPCDS queries
for external JDBC tables with Impala-Impala federation.
testdata/bin/create-tpc-jdbc-tables.py supports to create JDBC tables
for Impala-Impala, Postgres and MySQL.
Following sample commands creates TPCDS JDBC tables for Impala-Impala
federation with remote coordinator running at 10.19.10.86, and Postgres
server running at 10.19.10.86:
${IMPALA_HOME}/testdata/bin/create-tpc-jdbc-tables.py \
--jdbc_db_name=tpcds_jdbc --workload=tpcds \
--database_type=IMPALA --database_host=10.19.10.86 --clean
${IMPALA_HOME}/testdata/bin/create-tpc-jdbc-tables.py \
--jdbc_db_name=tpcds_jdbc --workload=tpcds \
--database_type=POSTGRES --database_host=10.19.10.86 \
--database_name=tpcds --clean
TODO
- run TPCDS queries in exhaustive mode.
- set proper default values for maxTotal and maxWaitMillis of DBCP
configuration parameters.
Remaining Issues:
- tpcds-decimal_v2-q80a failed with returned rows not matching
expected results for some decimal values.
- Coordinator open multiple JDBC connections in parallel for some
complex TPCDS quries with multiple DataSource scan nodes in
query plan, which cause connection not available from connection
pool if maxTotal is less than total number of scan nodes.
- maxWaitMillis is not working.
Testing:
- TODO Pass all TPCH/TCPDS queries.
Change-Id: I44e8c1bb020e90559c7f22483a7ab7a151b8f48a
---
M
fe/src/main/java/org/apache/impala/extdatasource/jdbc/conf/JdbcStorageConfigManager.java
M
fe/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java
M
fe/src/main/java/org/apache/impala/extdatasource/jdbc/dao/JdbcRecordIterator.java
M fe/src/main/java/org/apache/impala/planner/Planner.java
M testdata/bin/create-load-data.sh
A testdata/bin/create-tpc-jdbc-tables.py
A testdata/datasets/tpcds/tpcds_jdbc_schema_template.sql
A testdata/datasets/tpch/tpch_jdbc_schema_template.sql
M tests/query_test/test_tpcds_queries.py
M tests/query_test/test_tpch_queries.py
10 files changed, 1,532 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/21304/2
--
To view, visit http://gerrit.cloudera.org:8080/21304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I44e8c1bb020e90559c7f22483a7ab7a151b8f48a
Gerrit-Change-Number: 21304
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou
Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] WIP IMPALA-12910: Support running TPCH/TPCDS queries for JDBC tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21304 ) Change subject: WIP IMPALA-12910: Support running TPCH/TPCDS queries for JDBC tables .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15902/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I44e8c1bb020e90559c7f22483a7ab7a151b8f48a Gerrit-Change-Number: 21304 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Apr 2024 01:37:26 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-12910: Support running TPCH/TPCDS queries for JDBC tables
Wenzhe Zhou has uploaded this change for review. (
http://gerrit.cloudera.org:8080/21304
Change subject: WIP IMPALA-12910: Support running TPCH/TPCDS queries for JDBC
tables
..
WIP IMPALA-12910: Support running TPCH/TPCDS queries for JDBC tables
This patch adds script to create external JDBC tables for the dataset
of TPCH and TPCDS, and adds unit-tests to run TPCH and TPCDS queries
for external JDBC tables with Impala-Impala federation.
testdata/bin/create-tpc-jdbc-tables.py supports to create JDBC tables
for Impala-Impala, Postgres and MySQL.
Following sample commands creates TPCDS JDBC tables for Impala-Impala
federation with remote coordinator running at 10.19.10.86, and Postgres
server running at 10.19.10.86:
${IMPALA_HOME}/testdata/bin/create-tpc-jdbc-tables.py \
--jdbc_db_name=tpcds_jdbc --workload=tpcds \
--database_type=IMPALA --database_host=10.19.10.86 --clean
${IMPALA_HOME}/testdata/bin/create-tpc-jdbc-tables.py \
--jdbc_db_name=tpcds_jdbc --workload=tpcds \
--database_type=POSTGRES --database_host=10.19.10.86 \
--database_name=tpcds --clean
TODO
- run TPCDS queries in exhaustive mode.
- set proper default values for maxTotal and maxWaitMillis of DBCP
configuration parameters.
Remaining Issues:
- tpcds-decimal_v2-q80a failed with returned rows not matching
expected results for some decimal values.
- Coordinator open multiple JDBC connections in parallel for some
complex TPCDS quries with multiple DataSource scan nodes in
query plan, which cause connection not available from connection
pool if maxTotal is less than total number of scan nodes.
- maxWaitMillis is not working.
Testing:
- TODO Pass all TPCH/TCPDS queries.
Change-Id: I44e8c1bb020e90559c7f22483a7ab7a151b8f48a
---
M
fe/src/main/java/org/apache/impala/extdatasource/jdbc/conf/JdbcStorageConfigManager.java
M
fe/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java
M
fe/src/main/java/org/apache/impala/extdatasource/jdbc/dao/JdbcRecordIterator.java
M fe/src/main/java/org/apache/impala/planner/Planner.java
M testdata/bin/create-load-data.sh
A testdata/bin/create-tpc-jdbc-tables.py
A testdata/datasets/tpcds/tpcds_jdbc_schema_template.sql
A testdata/datasets/tpch/tpch_jdbc_schema_template.sql
M tests/query_test/test_tpcds_queries.py
M tests/query_test/test_tpch_queries.py
10 files changed, 1,527 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/21304/1
--
To view, visit http://gerrit.cloudera.org:8080/21304
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I44e8c1bb020e90559c7f22483a7ab7a151b8f48a
Gerrit-Change-Number: 21304
Gerrit-PatchSet: 1
Gerrit-Owner: Wenzhe Zhou
[Impala-ASF-CR] WIP IMPALA-12910: Support running TPCH/TPCDS queries for JDBC tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21304 ) Change subject: WIP IMPALA-12910: Support running TPCH/TPCDS queries for JDBC tables .. Patch Set 1: (6 comments) http://gerrit.cloudera.org:8080/#/c/21304/1/testdata/bin/create-tpc-jdbc-tables.py File testdata/bin/create-tpc-jdbc-tables.py: http://gerrit.cloudera.org:8080/#/c/21304/1/testdata/bin/create-tpc-jdbc-tables.py@41 PS1, Line 41: def drop_tables(): flake8: E302 expected 2 blank lines, found 1 http://gerrit.cloudera.org:8080/#/c/21304/1/testdata/bin/create-tpc-jdbc-tables.py@47 PS1, Line 47: def create_tables(): flake8: E302 expected 2 blank lines, found 1 http://gerrit.cloudera.org:8080/#/c/21304/1/testdata/bin/create-tpc-jdbc-tables.py@58 PS1, Line 58: flake8: E202 whitespace before '}' http://gerrit.cloudera.org:8080/#/c/21304/1/testdata/bin/create-tpc-jdbc-tables.py@74 PS1, Line 74: def get_test_file_path(workload): flake8: E302 expected 2 blank lines, found 1 http://gerrit.cloudera.org:8080/#/c/21304/1/testdata/bin/create-tpc-jdbc-tables.py@83 PS1, Line 83: if __name__ == "__main__": flake8: E305 expected 2 blank lines after class or function definition, found 1 http://gerrit.cloudera.org:8080/#/c/21304/1/tests/query_test/test_tpcds_queries.py File tests/query_test/test_tpcds_queries.py: http://gerrit.cloudera.org:8080/#/c/21304/1/tests/query_test/test_tpcds_queries.py@1119 PS1, Line 1119: d flake8: E301 expected 1 blank line, found 0 -- To view, visit http://gerrit.cloudera.org:8080/21304 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I44e8c1bb020e90559c7f22483a7ab7a151b8f48a Gerrit-Change-Number: 21304 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Apr 2024 01:14:23 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 ) Change subject: [WIP]Hierarchical metastore event processing .. Patch Set 9: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15892/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 9 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 15 Apr 2024 14:55:31 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21031 to look at the new patch set (#9). Change subject: [WIP]Hierarchical metastore event processing .. [WIP]Hierarchical metastore event processing Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/compat-apache-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/TableWriteId.java A fe/src/main/java/org/apache/impala/catalog/events/DBBarrierEvent.java A fe/src/main/java/org/apache/impala/catalog/events/DBEventExecutor.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java A fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/catalog/CatalogTableWriteIdTest.java A fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 16 files changed, 1,453 insertions(+), 57 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/21031/9 -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 9 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/21031 )
Change subject: [WIP]Hierarchical metastore event processing
..
Patch Set 9:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/21031/9/fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java
File
fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java:
http://gerrit.cloudera.org:8080/#/c/21031/9/fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java@269
PS9, Line 269: "[Performance] Non-Partitioned transactional table with
hierarchical processing: {}",
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/21031/9/fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java@272
PS9, Line 272: "tblproperties ('transactional'='true',
'transactional_properties'='insert_only')");
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/21031/9/fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java@289
PS9, Line 289: "tblproperties ('transactional'='true',
'transactional_properties'='insert_only')");
line too long (92 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/21031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6
Gerrit-Change-Number: 21031
Gerrit-PatchSet: 9
Gerrit-Owner: Anonymous Coward
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Mon, 15 Apr 2024 14:32:27 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP: Bump Impyla version to 0.20a1
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21286 ) Change subject: WIP: Bump Impyla version to 0.20a1 .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/21286 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I297a2a34f3e688555ce8572b6c7fffbd34423f2d Gerrit-Change-Number: 21286 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 11 Apr 2024 12:57:49 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: Bump Impyla version to 0.20a1
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21286 ) Change subject: WIP: Bump Impyla version to 0.20a1 .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15868/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21286 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I297a2a34f3e688555ce8572b6c7fffbd34423f2d Gerrit-Change-Number: 21286 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 11 Apr 2024 08:16:42 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: Bump Impyla version to 0.20a1
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21286 to look at the new patch set (#2). Change subject: WIP: Bump Impyla version to 0.20a1 .. WIP: Bump Impyla version to 0.20a1 Change-Id: I297a2a34f3e688555ce8572b6c7fffbd34423f2d --- M infra/python/deps/requirements.txt M tests/common/impala_connection.py 2 files changed, 3 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/21286/2 -- To view, visit http://gerrit.cloudera.org:8080/21286 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I297a2a34f3e688555ce8572b6c7fffbd34423f2d Gerrit-Change-Number: 21286 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] WIP: Bump Impyla version to 0.20a1
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21286 ) Change subject: WIP: Bump Impyla version to 0.20a1 .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10531/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/21286 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I297a2a34f3e688555ce8572b6c7fffbd34423f2d Gerrit-Change-Number: 21286 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 11 Apr 2024 07:52:48 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: Bump Impyla version to 0.20a1
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21286 ) Change subject: WIP: Bump Impyla version to 0.20a1 .. Patch Set 1: Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/10526/ -- To view, visit http://gerrit.cloudera.org:8080/21286 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I297a2a34f3e688555ce8572b6c7fffbd34423f2d Gerrit-Change-Number: 21286 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 11 Apr 2024 03:39:16 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: Bump Impyla version to 0.20a1
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21286 ) Change subject: WIP: Bump Impyla version to 0.20a1 .. Patch Set 1: Verified-1 Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/15859/ : ABORTED -- To view, visit http://gerrit.cloudera.org:8080/21286 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I297a2a34f3e688555ce8572b6c7fffbd34423f2d Gerrit-Change-Number: 21286 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 10 Apr 2024 20:37:43 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: Bump Impyla version to 0.20a1
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21286 ) Change subject: WIP: Bump Impyla version to 0.20a1 .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10526/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/21286 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I297a2a34f3e688555ce8572b6c7fffbd34423f2d Gerrit-Change-Number: 21286 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 10 Apr 2024 17:38:14 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: Bump Impyla version to 0.20a1
Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21286 Change subject: WIP: Bump Impyla version to 0.20a1 .. WIP: Bump Impyla version to 0.20a1 Change-Id: I297a2a34f3e688555ce8572b6c7fffbd34423f2d --- M infra/python/deps/requirements.txt M tests/common/impala_connection.py 2 files changed, 3 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/21286/1 -- To view, visit http://gerrit.cloudera.org:8080/21286 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I297a2a34f3e688555ce8572b6c7fffbd34423f2d Gerrit-Change-Number: 21286 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer
[Impala-ASF-CR] WIP IMPALA-12983 (Part 1): Push down simple limit to remote database
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21281 ) Change subject: WIP IMPALA-12983 (Part 1): Push down simple limit to remote database .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15846/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21281 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I139d34cc3b70db14d5f45266d2cbd586a53c8382 Gerrit-Change-Number: 21281 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 10 Apr 2024 06:52:18 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-12983 (Part 1): Push down simple limit to remote database
Wenzhe Zhou has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21281 Change subject: WIP IMPALA-12983 (Part 1): Push down simple limit to remote database .. WIP IMPALA-12983 (Part 1): Push down simple limit to remote database This patch pushes down the simple limit on DataSourceScanNode to remote database. Simple limit means the scan node does not have scan conjunct. Testing: - TODO: Add unit-test to verify RowsRead of DataSourceNode in query profiles, which are the rows read from JDBC driver. - Passed end-to-end unit tests for external JDBC tables Change-Id: I139d34cc3b70db14d5f45266d2cbd586a53c8382 --- M be/src/exec/data-source-scan-node.cc M common/thrift/PlanNodes.thrift M fe/src/main/java/org/apache/impala/planner/DataSourceScanNode.java 3 files changed, 9 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/21281/1 -- To view, visit http://gerrit.cloudera.org:8080/21281 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I139d34cc3b70db14d5f45266d2cbd586a53c8382 Gerrit-Change-Number: 21281 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou
[Impala-ASF-CR] WIP IMPALA-12362: (part-4/4) Refactor linux packaging related cmake files.
Quanlong Huang has posted comments on this change. (
http://gerrit.cloudera.org:8080/20263 )
Change subject: WIP IMPALA-12362: (part-4/4) Refactor linux packaging related
cmake files.
..
Patch Set 7:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20263/7/package/CMakeLists.txt
File package/CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/20263/7/package/CMakeLists.txt@39
PS7, Line 39: FILE(GLOB gcc_lib ${IMPALA_GCC_HOME}/lib64/libgcc_s.so.1*)
: install(FILES ${gcc_lib} DESTINATION
${IMPALA_INSTALLDIR}/lib/native)
:
: FILE(GLOB cpp_lib ${IMPALA_GCC_HOME}/lib64/libstdc++.so.6*)
: install(FILES ${cpp_lib} DESTINATION
${IMPALA_INSTALLDIR}/lib/native)
:
: set(KUDU_HOME
$ENV{IMPALA_TOOLCHAIN_PACKAGES_HOME}/kudu-$ENV{IMPALA_KUDU_VERSION}/release)
: # The parent folder is lib64 on centos/redhat, while on ubuntu
it's lib.
: FILE(GLOB kudu_lib ${KUDU_HOME}/lib*/libkudu_client.so*)
: install(FILES ${kudu_lib} DESTINATION
${IMPALA_INSTALLDIR}/lib/native)
:
: FILE(GLOB hadoop_lib $ENV{HADOOP_LIB_DIR}/native/libhadoop.so*)
: install(FILES ${hadoop_lib} DESTINATION
${IMPALA_INSTALLDIR}/lib/native)
Can we add a check that if any of these files are not found, exit the build? I
encountered such issue when backporting the packaging support to older branches.
--
To view, visit http://gerrit.cloudera.org:8080/20263
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If3914dcda69f81a735cdf70d76c59fa09454777b
Gerrit-Change-Number: 20263
Gerrit-PatchSet: 7
Gerrit-Owner: Xiang Yang
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Laszlo Gaal
Gerrit-Reviewer: Quanlong Huang
Gerrit-Reviewer: Xiang Yang
Gerrit-Comment-Date: Mon, 08 Apr 2024 05:28:00 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP IMPALA-12362: (part-3/4) Add more binaries to packaging module.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20929 ) Change subject: WIP IMPALA-12362: (part-3/4) Add more binaries to packaging module. .. Patch Set 3: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/20929 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I594742037a05d4d74d6a2bc011619713f7ca12e4 Gerrit-Change-Number: 20929 Gerrit-PatchSet: 3 Gerrit-Owner: Xiang Yang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Sun, 07 Apr 2024 11:41:53 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 ) Change subject: [WIP]Hierarchical metastore event processing .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15758/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 8 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 02 Apr 2024 12:00:00 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 ) Change subject: [WIP]Hierarchical metastore event processing .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15757/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 7 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 02 Apr 2024 11:56:23 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21031 to look at the new patch set (#8). Change subject: [WIP]Hierarchical metastore event processing .. [WIP]Hierarchical metastore event processing Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/compat-apache-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/TableWriteId.java A fe/src/main/java/org/apache/impala/catalog/events/DBBarrierEvent.java A fe/src/main/java/org/apache/impala/catalog/events/DBEventExecutor.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java A fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/catalog/CatalogTableWriteIdTest.java A fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 16 files changed, 1,339 insertions(+), 57 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/21031/8 -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 8 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 ) Change subject: [WIP]Hierarchical metastore event processing .. Patch Set 8: (2 comments) http://gerrit.cloudera.org:8080/#/c/21031/8/fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java File fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java: http://gerrit.cloudera.org:8080/#/c/21031/8/fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java@58 PS8, Line 58: private static final Logger LOG = LoggerFactory.getLogger(EventsProcessorPerfTest.class); line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/21031/8/fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java@76 PS8, Line 76: prevHierarchicalProcessing_ = BackendConfig.INSTANCE.isHierarchicalEventProcessingEnabled(); line too long (96 > 90) -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 8 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 02 Apr 2024 11:37:34 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21031 to look at the new patch set (#7). Change subject: [WIP]Hierarchical metastore event processing .. [WIP]Hierarchical metastore event processing Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/compat-apache-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/TableWriteId.java A fe/src/main/java/org/apache/impala/catalog/events/DBBarrierEvent.java A fe/src/main/java/org/apache/impala/catalog/events/DBEventExecutor.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java A fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/catalog/CatalogTableWriteIdTest.java A fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 16 files changed, 1,342 insertions(+), 57 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/21031/7 -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 7 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 ) Change subject: [WIP]Hierarchical metastore event processing .. Patch Set 7: (2 comments) http://gerrit.cloudera.org:8080/#/c/21031/7/fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java File fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java: http://gerrit.cloudera.org:8080/#/c/21031/7/fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java@61 PS7, Line 61: private static final Logger LOG = LoggerFactory.getLogger(EventsProcessorPerfTest.class); line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/21031/7/fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorPerfTest.java@79 PS7, Line 79: prevHierarchicalProcessing_ = BackendConfig.INSTANCE.isHierarchicalEventProcessingEnabled(); line too long (96 > 90) -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 7 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 02 Apr 2024 11:34:02 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP IMPALA-12362: (part-2/4) Optimize default configurations for packaging module.
Xiang Yang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20928 ) Change subject: WIP IMPALA-12362: (part-2/4) Optimize default configurations for packaging module. .. Patch Set 4: Hi zihao, Can you go ahead and help me review the second part of IMPALA-12362? many thanks:) -- To view, visit http://gerrit.cloudera.org:8080/20928 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifda229b779a3d6fca647bb81fe23dd61ad7e5d66 Gerrit-Change-Number: 20928 Gerrit-PatchSet: 4 Gerrit-Owner: Xiang Yang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Xiang Yang Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Tue, 02 Apr 2024 02:06:42 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 ) Change subject: [WIP]Hierarchical metastore event processing .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15737/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 6 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 29 Mar 2024 14:51:34 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21031 to look at the new patch set (#6). Change subject: [WIP]Hierarchical metastore event processing .. [WIP]Hierarchical metastore event processing Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/compat-apache-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/TableWriteId.java A fe/src/main/java/org/apache/impala/catalog/events/DBBarrierEvent.java A fe/src/main/java/org/apache/impala/catalog/events/DBEventExecutor.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java A fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/catalog/CatalogTableWriteIdTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 14 files changed, 1,067 insertions(+), 57 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/21031/6 -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 6 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 ) Change subject: [WIP]Hierarchical metastore event processing .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15736/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 5 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 29 Mar 2024 14:24:19 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21031 to look at the new patch set (#5). Change subject: [WIP]Hierarchical metastore event processing .. [WIP]Hierarchical metastore event processing Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/compat-apache-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/TableWriteId.java A fe/src/main/java/org/apache/impala/catalog/events/DBBarrierEvent.java A fe/src/main/java/org/apache/impala/catalog/events/DBEventExecutor.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java A fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/catalog/CatalogTableWriteIdTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 14 files changed, 1,067 insertions(+), 57 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/21031/5 -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 5 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21031 ) Change subject: [WIP]Hierarchical metastore event processing .. Patch Set 4: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/15735/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 4 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 29 Mar 2024 13:50:53 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21031 to look at the new patch set (#4). Change subject: [WIP]Hierarchical metastore event processing .. [WIP]Hierarchical metastore event processing Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/compat-apache-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/TableWriteId.java A fe/src/main/java/org/apache/impala/catalog/events/DBBarrierEvent.java A fe/src/main/java/org/apache/impala/catalog/events/DBEventExecutor.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java A fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/catalog/CatalogTableWriteIdTest.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 14 files changed, 1,018 insertions(+), 57 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/21031/4 -- To view, visit http://gerrit.cloudera.org:8080/21031 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6 Gerrit-Change-Number: 21031 Gerrit-PatchSet: 4 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [WIP]Hierarchical metastore event processing
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/21031 )
Change subject: [WIP]Hierarchical metastore event processing
..
Patch Set 4:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/21031/4/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
File fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java:
http://gerrit.cloudera.org:8080/#/c/21031/4/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java@1037
PS4, Line 1037: + "processing cannot continue. Issue an
invalidate metadata command to reset "
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/21031/4/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java@1083
PS4, Line 1083: + "get write event infos for txn %d. Event
processing cannot continue. Issue "
line too long (96 > 90)
http://gerrit.cloudera.org:8080/#/c/21031/4/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java:
http://gerrit.cloudera.org:8080/#/c/21031/4/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@3223
PS4, Line 3223: + "mark aborted write ids to table for txn
{}. Event processing cannot "
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/21031/4/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@3224
PS4, Line 3224: + "continue. Issue an invalidate metadata
command to reset event processor.",
line too long (97 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/21031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I76d8a739f9db6d40f01028bfd786a85d83f9e5d6
Gerrit-Change-Number: 21031
Gerrit-PatchSet: 4
Gerrit-Owner: Anonymous Coward
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Fri, 29 Mar 2024 13:28:04 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP IMPALA-12933: Avoid fetching unneccessary event types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21186 ) Change subject: WIP IMPALA-12933: Avoid fetching unneccessary event types .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15644/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieabe714328aa2cc605cb62b85ae8aa4bd537dbe9 Gerrit-Change-Number: 21186 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Sat, 23 Mar 2024 00:18:58 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-12933: Avoid fetching unneccessary event types
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21186 to look at the new patch set (#2). Change subject: WIP IMPALA-12933: Avoid fetching unneccessary event types .. WIP IMPALA-12933: Avoid fetching unneccessary event types There are several places that catalogd will fetch all events of a specifit type on a table. E.g. in TableLoader#load(), if the table has an old createEventId, catalogd will fetch all CREATE_TABLE events after that createEventId on the table. Fetching the list of events is expensive since the filtering is done on client side, i.e. catalogd fetches all events and filter them locally based on the event type and table name. This could take hours if there are lots of events (e.g 1M) in HMS. This patch sets the eventTypeSkipList with the complement set of the wanted type. So the get_next_notification RPC can filter out some events on HMS side. Also adds UPDATE_PART_COL_STAT_EVENT to the default skip list. Change-Id: Ieabe714328aa2cc605cb62b85ae8aa4bd537dbe9 --- M fe/src/compat-apache-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/TableLoader.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServiceHandler.java M fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 8 files changed, 138 insertions(+), 48 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/21186/2 -- To view, visit http://gerrit.cloudera.org:8080/21186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ieabe714328aa2cc605cb62b85ae8aa4bd537dbe9 Gerrit-Change-Number: 21186 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] WIP IMPALA-5323: Support Kudu BINARY
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18868 ) Change subject: WIP IMPALA-5323: Support Kudu BINARY .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/10415/ -- To view, visit http://gerrit.cloudera.org:8080/18868 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iff701a4b3a09ce7b6982c5d238e65f3d4f3d1151 Gerrit-Change-Number: 18868 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Fri, 22 Mar 2024 23:10:21 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-12933: Avoid fetching unneccessary event types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21186 ) Change subject: WIP IMPALA-12933: Avoid fetching unneccessary event types .. Patch Set 1: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/10411/ -- To view, visit http://gerrit.cloudera.org:8080/21186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieabe714328aa2cc605cb62b85ae8aa4bd537dbe9 Gerrit-Change-Number: 21186 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 22 Mar 2024 18:48:21 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-5323: Support Kudu BINARY
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18868 ) Change subject: WIP IMPALA-5323: Support Kudu BINARY .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15632/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18868 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iff701a4b3a09ce7b6982c5d238e65f3d4f3d1151 Gerrit-Change-Number: 18868 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Fri, 22 Mar 2024 18:21:54 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-5323: Support Kudu BINARY
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/18868 )
Change subject: WIP IMPALA-5323: Support Kudu BINARY
..
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18868/3/be/src/exprs/expr-test-common.h
File be/src/exprs/expr-test-common.h:
http://gerrit.cloudera.org:8080/#/c/18868/3/be/src/exprs/expr-test-common.h@552
PS3, Line 552: void ExprTestBase::TestTimestampValue(const string& expr, const
TimestampValue& expected_result) {
line too long (98 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/18868
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iff701a4b3a09ce7b6982c5d238e65f3d4f3d1151
Gerrit-Change-Number: 18868
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Peter Rozsa
Gerrit-Comment-Date: Fri, 22 Mar 2024 17:59:27 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP IMPALA-5323: Support Kudu BINARY
Hello Daniel Becker, Peter Rozsa, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18868 to look at the new patch set (#3). Change subject: WIP IMPALA-5323: Support Kudu BINARY .. WIP IMPALA-5323: Support Kudu BINARY Change-Id: Iff701a4b3a09ce7b6982c5d238e65f3d4f3d1151 --- M be/src/exec/kudu/kudu-table-sink.cc M be/src/exec/kudu/kudu-util-ir.cc M be/src/exec/kudu/kudu-util.cc M be/src/exprs/cast-functions-ir.cc A be/src/exprs/expr-test-common.h M be/src/runtime/types.cc M be/src/runtime/types.h M fe/src/main/java/org/apache/impala/analysis/CastExpr.java M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java M fe/src/main/java/org/apache/impala/util/KuduUtil.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M tests/query_test/test_scanners.py 13 files changed, 666 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/18868/3 -- To view, visit http://gerrit.cloudera.org:8080/18868 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iff701a4b3a09ce7b6982c5d238e65f3d4f3d1151 Gerrit-Change-Number: 18868 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] WIP IMPALA-5323: Support Kudu BINARY
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18868 ) Change subject: WIP IMPALA-5323: Support Kudu BINARY .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10415/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/18868 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iff701a4b3a09ce7b6982c5d238e65f3d4f3d1151 Gerrit-Change-Number: 18868 Gerrit-PatchSet: 3 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Fri, 22 Mar 2024 17:59:36 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-12933: Avoid fetching unneccessary event types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21186 ) Change subject: WIP IMPALA-12933: Avoid fetching unneccessary event types .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15625/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieabe714328aa2cc605cb62b85ae8aa4bd537dbe9 Gerrit-Change-Number: 21186 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 22 Mar 2024 14:11:51 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-12933: Avoid fetching unneccessary event types
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21186 Change subject: WIP IMPALA-12933: Avoid fetching unneccessary event types .. WIP IMPALA-12933: Avoid fetching unneccessary event types There are several places that catalogd will fetch all events of a specifit type on a table. E.g. in TableLoader#load(), if the table has an old createEventId, catalogd will fetch all CREATE_TABLE events after that createEventId on the table. Fetching the list of events is expensive since the filtering is done on client side, i.e. catalogd fetches all events and filter them locally based on the event type and table name. This could take hours if there are lots of events (e.g 1M) in HMS. This patch sets the eventTypeSkipList with the complement set of the wanted type. So the get_next_notification RPC can filter out some events on HMS side. Change-Id: Ieabe714328aa2cc605cb62b85ae8aa4bd537dbe9 --- M fe/src/main/java/org/apache/impala/catalog/TableLoader.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServiceHandler.java M fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 6 files changed, 128 insertions(+), 40 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/21186/1 -- To view, visit http://gerrit.cloudera.org:8080/21186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ieabe714328aa2cc605cb62b85ae8aa4bd537dbe9 Gerrit-Change-Number: 21186 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang
[Impala-ASF-CR] WIP IMPALA-12933: Avoid fetching unneccessary event types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21186 ) Change subject: WIP IMPALA-12933: Avoid fetching unneccessary event types .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10411/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/21186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieabe714328aa2cc605cb62b85ae8aa4bd537dbe9 Gerrit-Change-Number: 21186 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 22 Mar 2024 13:50:31 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-12915: Use libgtest.so when built with shared libs
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/21163 ) Change subject: WIP IMPALA-12915: Use libgtest.so when built with shared libs .. Patch Set 2: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/21163/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21163/2//COMMIT_MSG@7 PS2, Line 7: WIP nit: drop WIP? -- To view, visit http://gerrit.cloudera.org:8080/21163 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I27d21217db219f52b072a4e5cfa1caaace35d1a2 Gerrit-Change-Number: 21163 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Thu, 21 Mar 2024 04:03:03 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [WIP] IMPALA-12832: Test jenkins tests
Sai Hemanth Gantasala has abandoned this change. ( http://gerrit.cloudera.org:8080/21069 ) Change subject: [WIP] IMPALA-12832: Test jenkins tests .. Abandoned Patch is already merged via another gerrit -- To view, visit http://gerrit.cloudera.org:8080/21069 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I1d7ab7c46e039b0a23d9b998eff7940bce763b1d Gerrit-Change-Number: 21069 Gerrit-PatchSet: 1 Gerrit-Owner: Sai Hemanth Gantasala Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] WIP IMPALA-12915: Use libgtest.so when built with shared libs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21163 ) Change subject: WIP IMPALA-12915: Use libgtest.so when built with shared libs .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15568/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/21163 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I27d21217db219f52b072a4e5cfa1caaace35d1a2 Gerrit-Change-Number: 21163 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 19 Mar 2024 22:17:24 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-12915: Use libgtest.so when built with shared libs
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21163 to look at the new patch set (#2). Change subject: WIP IMPALA-12915: Use libgtest.so when built with shared libs .. WIP IMPALA-12915: Use libgtest.so when built with shared libs This fixes the double-free memory issue when running unifiedbetests built with shared libs Change-Id: I27d21217db219f52b072a4e5cfa1caaace35d1a2 --- M CMakeLists.txt M bin/impala-config.sh M cmake_modules/FindGTest.cmake 3 files changed, 19 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/21163/2 -- To view, visit http://gerrit.cloudera.org:8080/21163 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I27d21217db219f52b072a4e5cfa1caaace35d1a2 Gerrit-Change-Number: 21163 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] WIP IMPALA-12915: Use libgtest.so when built with shared libs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21163 ) Change subject: WIP IMPALA-12915: Use libgtest.so when built with shared libs .. Patch Set 1: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/15557/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/21163 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I27d21217db219f52b072a4e5cfa1caaace35d1a2 Gerrit-Change-Number: 21163 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 19 Mar 2024 11:52:58 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-12915: Use libgtest.so when built with shared libs
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21163 Change subject: WIP IMPALA-12915: Use libgtest.so when built with shared libs .. WIP IMPALA-12915: Use libgtest.so when built with shared libs This fixes the double-free memory issue when running unifiedbetests built with shared libs Change-Id: I27d21217db219f52b072a4e5cfa1caaace35d1a2 --- M CMakeLists.txt M bin/impala-config.sh M cmake_modules/FindGTest.cmake 3 files changed, 19 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/21163/1 -- To view, visit http://gerrit.cloudera.org:8080/21163 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I27d21217db219f52b072a4e5cfa1caaace35d1a2 Gerrit-Change-Number: 21163 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang
[Impala-ASF-CR] [WIP] IMPALA-12856: Event processor should ignore processing partition with empty partition values
Quanlong Huang has posted comments on this change. (
http://gerrit.cloudera.org:8080/21143 )
Change subject: [WIP] IMPALA-12856: Event processor should ignore processing
partition with empty partition values
..
Patch Set 1:
(6 comments)
LGTM, just have some minor comments to make this a FE-only change.
http://gerrit.cloudera.org:8080/#/c/21143/1/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:
http://gerrit.cloudera.org:8080/#/c/21143/1/be/src/catalog/catalog-server.cc@192
PS1, Line 192: DEFINE_bool_hidden(is_return_empty_partition_values, false,
"This configuration is used "
We don't need this for FE tests. We can use the debug_actions flag and add an
action appropriately.
http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java:
http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@2846
PS1, Line 2846: public int reloadPartitionsFromNames(IMetaStoreClient client,
nit: it seems this is only used by the event-processor. We can pass in the
eventId and eventType to improve the logging if that's not too hard.
http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@2858
PS1, Line 2858: warn
nit: error() seems more suitable. This can also be simplified as
LOG.error("Received partition with empty values: {}. \nIgnoring" +
" reloading the partition.", partition));
http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@2891
PS1, Line 2891: public int reloadPartitionsFromEvent(IMetaStoreClient client,
nit: we can pass in the eventId and eventType to improve the logging
http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/service/BackendConfig.java@461
PS1, Line 461: public String debugActions() { return
backendCfg_.debug_actions; }
We can add setDebugAction() for the new FE test.
http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java
File fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java:
http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java@212
PS1, Line 212: if
(BackendConfig.INSTANCE.getIsReturnEmptyPartitionValues()) {
and use the debug action here like
DebugUtils.hasDebugAction(BackendConfig.INSTANCE.debugActions(),
DebugUtils.MOCK_EMPTY_PARTITION_VALUES)
--
To view, visit http://gerrit.cloudera.org:8080/21143
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id2469930ccd74948325f1723bd8b2bd6aad02d09
Gerrit-Change-Number: 21143
Gerrit-PatchSet: 1
Gerrit-Owner: Sai Hemanth Gantasala
Gerrit-Reviewer: Anonymous Coward
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Quanlong Huang
Gerrit-Reviewer: Sai Hemanth Gantasala
Gerrit-Comment-Date: Wed, 13 Mar 2024 12:44:09 +
Gerrit-HasComments: Yes
