[Impala-ASF-CR] IMPALA-11366: Fix impala-shell build failure if the build date contains Non-ASCII characters
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18632 ) Change subject: IMPALA-11366: Fix impala-shell build failure if the build date contains Non-ASCII characters .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18632 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I596121e08a52a4bc6d8668cf7e8b61b6c34eb4b9 Gerrit-Change-Number: 18632 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Mon, 20 Jun 2022 07:06:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11356: Fix test sfs.py to account for new Hive behavior
Hello Fang-Yu Rao, Joe McDonnell, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18618 to look at the new patch set (#2). Change subject: IMPALA-11356: Fix test_sfs.py to account for new Hive behavior .. IMPALA-11356: Fix test_sfs.py to account for new Hive behavior Hive changed behavior in HIVE-24920 to allow external SFS tables to point to locations in managed tables. test_sfs.py had a test case for the previous failure, so it broke with this change. This changes test_sfs.py to verify the new behavior where an external SFS table can point to a managed location. Testing: - Ran test_sfs.py locally with recent GBN Change-Id: I426aacdba7afba3f3a747a4e02632d15e38c63c9 --- M testdata/workloads/functional-query/queries/QueryTest/sfs.test 1 file changed, 63 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/18618/2 -- To view, visit http://gerrit.cloudera.org:8080/18618 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I426aacdba7afba3f3a747a4e02632d15e38c63c9 Gerrit-Change-Number: 18618 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-11365: Dereferencing null pointer in TopNNode
Daniel Becker has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/18629 ) Change subject: IMPALA-11365: Dereferencing null pointer in TopNNode .. IMPALA-11365: Dereferencing null pointer in TopNNode In the constructor of TopNNode, if 'pnode.partition_comparator_config_' is NULL, we initialise 'partition_cmp_' with a NULL pointer. However, when initialising 'partition_heaps_', we dereference 'partition_cmp_' because 'ComparatorWrapper' expects a reference. This has so far not lead to a crash because in this case the comparator of 'partition_heaps_' is not used, but assigning a NULL pointer to a reference is undefined behaviour. After this change, instead of assigning a NULL pointer to 'partition_cmp_', we use a dummy comparator, and no undefined behaviour is invoked. Change-Id: I0b15b06f608b4d17fdf8a24e05967aaa16ebb79c --- M be/src/exec/topn-node.cc M be/src/util/tuple-row-compare.h 2 files changed, 32 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/18629/2 -- To view, visit http://gerrit.cloudera.org:8080/18629 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I0b15b06f608b4d17fdf8a24e05967aaa16ebb79c Gerrit-Change-Number: 18629 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-11366: Fix impala-shell build failure if the build date contains Non-ASCII characters
lipeng...@sensorsdata.cn has posted comments on this change. ( http://gerrit.cloudera.org:8080/18632 ) Change subject: IMPALA-11366: Fix impala-shell build failure if the build date contains Non-ASCII characters .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/18632 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I596121e08a52a4bc6d8668cf7e8b61b6c34eb4b9 Gerrit-Change-Number: 18632 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Mon, 20 Jun 2022 08:52:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11356: Fix test sfs.py to account for new Hive behavior
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18618 ) Change subject: IMPALA-11356: Fix test_sfs.py to account for new Hive behavior .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18618 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I426aacdba7afba3f3a747a4e02632d15e38c63c9 Gerrit-Change-Number: 18618 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Mon, 20 Jun 2022 08:57:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11349: Bump CDP BUILD NUMBER to 27992803
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18613 ) Change subject: IMPALA-11349: Bump CDP_BUILD_NUMBER to 27992803 .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18613 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9fcef254a5f845540e09e0d8d2111f305fb2fea2 Gerrit-Change-Number: 18613 Gerrit-PatchSet: 1 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Mon, 20 Jun 2022 08:57:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11365: Dereferencing null pointer in TopNNode
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18629 ) Change subject: IMPALA-11365: Dereferencing null pointer in TopNNode .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10811/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18629 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0b15b06f608b4d17fdf8a24e05967aaa16ebb79c Gerrit-Change-Number: 18629 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 20 Jun 2022 09:03:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11356: Fix test sfs.py to account for new Hive behavior
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18618 ) Change subject: IMPALA-11356: Fix test_sfs.py to account for new Hive behavior .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8247/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/18618 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I426aacdba7afba3f3a747a4e02632d15e38c63c9 Gerrit-Change-Number: 18618 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Mon, 20 Jun 2022 09:15:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11367: Fix some formatting bugs when DESCRIBE FORMATTED for Iceberg tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18634 ) Change subject: IMPALA-11367: Fix some formatting bugs when DESCRIBE FORMATTED for Iceberg tables .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8248/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/18634 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I991644cefb34decc843a5542b47eaec11d7b6e42 Gerrit-Change-Number: 18634 Gerrit-PatchSet: 5 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 20 Jun 2022 09:24:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11367: Fix some formatting bugs when DESCRIBE FORMATTED for Iceberg tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18634 ) Change subject: IMPALA-11367: Fix some formatting bugs when DESCRIBE FORMATTED for Iceberg tables .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18634 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I991644cefb34decc843a5542b47eaec11d7b6e42 Gerrit-Change-Number: 18634 Gerrit-PatchSet: 5 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 20 Jun 2022 09:24:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11358: Fixed Kudu table's missing comment
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18627 ) Change subject: IMPALA-11358: Fixed Kudu table's missing comment .. Patch Set 8: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18627 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf66f8b4679b00da6693a27fed79b04e8f6afb55 Gerrit-Change-Number: 18627 Gerrit-PatchSet: 8 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 20 Jun 2022 09:26:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11358: Fixed Kudu table's missing comment
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18627 ) Change subject: IMPALA-11358: Fixed Kudu table's missing comment .. Patch Set 8: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8249/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/18627 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf66f8b4679b00da6693a27fed79b04e8f6afb55 Gerrit-Change-Number: 18627 Gerrit-PatchSet: 8 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 20 Jun 2022 09:26:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11367: Fix some formatting bugs when DESCRIBE FORMATTED for Iceberg tables
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/18634 ) Change subject: IMPALA-11367: Fix some formatting bugs when DESCRIBE FORMATTED for Iceberg tables .. Patch Set 4: Code-Review+2 LGTM! -- To view, visit http://gerrit.cloudera.org:8080/18634 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I991644cefb34decc843a5542b47eaec11d7b6e42 Gerrit-Change-Number: 18634 Gerrit-PatchSet: 4 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 20 Jun 2022 09:23:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11358: Fixed Kudu table's missing comment
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/18627 ) Change subject: IMPALA-11358: Fixed Kudu table's missing comment .. Patch Set 7: Code-Review+2 LGTM! Thanks for fixing this. -- To view, visit http://gerrit.cloudera.org:8080/18627 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf66f8b4679b00da6693a27fed79b04e8f6afb55 Gerrit-Change-Number: 18627 Gerrit-PatchSet: 7 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 20 Jun 2022 09:25:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11365: Dereferencing null pointer in TopNNode
Daniel Becker has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/18629 ) Change subject: IMPALA-11365: Dereferencing null pointer in TopNNode .. IMPALA-11365: Dereferencing null pointer in TopNNode In the constructor of TopNNode, if 'pnode.partition_comparator_config_' is NULL, we initialise 'partition_cmp_' with a NULL pointer. However, when initialising 'partition_heaps_', we dereference 'partition_cmp_' because 'ComparatorWrapper' expects a reference. This has so far not lead to a crash because in this case the comparator of 'partition_heaps_' is not used, but assigning a NULL pointer to a reference is undefined behaviour. After this change, instead of assigning a NULL pointer to 'partition_cmp_', we use a dummy comparator, and no undefined behaviour is invoked. Change-Id: I0b15b06f608b4d17fdf8a24e05967aaa16ebb79c --- M be/src/exec/topn-node.cc M be/src/util/tuple-row-compare.h 2 files changed, 34 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/18629/3 -- To view, visit http://gerrit.cloudera.org:8080/18629 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I0b15b06f608b4d17fdf8a24e05967aaa16ebb79c Gerrit-Change-Number: 18629 Gerrit-PatchSet: 3 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-11365: Dereferencing null pointer in TopNNode
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18629 ) Change subject: IMPALA-11365: Dereferencing null pointer in TopNNode .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10812/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18629 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0b15b06f608b4d17fdf8a24e05967aaa16ebb79c Gerrit-Change-Number: 18629 Gerrit-PatchSet: 3 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 20 Jun 2022 10:14:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11366: Fix impala-shell build failure if the build date contains Non-ASCII characters
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18632 ) Change subject: IMPALA-11366: Fix impala-shell build failure if the build date contains Non-ASCII characters .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8250/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/18632 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I596121e08a52a4bc6d8668cf7e8b61b6c34eb4b9 Gerrit-Change-Number: 18632 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Mon, 20 Jun 2022 12:32:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11366: Fix impala-shell build failure if the build date contains Non-ASCII characters
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18632 ) Change subject: IMPALA-11366: Fix impala-shell build failure if the build date contains Non-ASCII characters .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18632 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I596121e08a52a4bc6d8668cf7e8b61b6c34eb4b9 Gerrit-Change-Number: 18632 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Mon, 20 Jun 2022 12:32:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11233: Unset all query option
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18430 ) Change subject: IMPALA-11233: Unset all query option .. Patch Set 13: Merging this. Thanks for your contribution, Xiaoqing! -- To view, visit http://gerrit.cloudera.org:8080/18430 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iabf23622daab733ddab20dd3ca73af6c9bd5c250 Gerrit-Change-Number: 18430 Gerrit-PatchSet: 13 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jian Zhang Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Xiaoqing Gao Gerrit-Comment-Date: Mon, 20 Jun 2022 12:35:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11233: Unset all query option
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18430 ) Change subject: IMPALA-11233: Unset all query option .. Patch Set 13: Code-Review+2 > Patch Set 13: > > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8240/ The failure is due to IMPALA-11160 and is unrelated to this patch: https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5864 -- To view, visit http://gerrit.cloudera.org:8080/18430 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iabf23622daab733ddab20dd3ca73af6c9bd5c250 Gerrit-Change-Number: 18430 Gerrit-PatchSet: 13 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jian Zhang Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Xiaoqing Gao Gerrit-Comment-Date: Mon, 20 Jun 2022 12:35:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11233: Unset all query option
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18430 ) Change subject: IMPALA-11233: Unset all query option .. Patch Set 14: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8251/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/18430 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iabf23622daab733ddab20dd3ca73af6c9bd5c250 Gerrit-Change-Number: 18430 Gerrit-PatchSet: 14 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jian Zhang Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Xiaoqing Gao Gerrit-Comment-Date: Mon, 20 Jun 2022 12:36:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11233: Unset all query option
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18430 ) Change subject: IMPALA-11233: Unset all query option .. Patch Set 14: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18430 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iabf23622daab733ddab20dd3ca73af6c9bd5c250 Gerrit-Change-Number: 18430 Gerrit-PatchSet: 14 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jian Zhang Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Xiaoqing Gao Gerrit-Comment-Date: Mon, 20 Jun 2022 12:36:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11356: Fix test sfs.py to account for new Hive behavior
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18618 ) Change subject: IMPALA-11356: Fix test_sfs.py to account for new Hive behavior .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/18618 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I426aacdba7afba3f3a747a4e02632d15e38c63c9 Gerrit-Change-Number: 18618 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Mon, 20 Jun 2022 13:43:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11349: Bump CDP BUILD NUMBER to 27992803
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/18613 ) Change subject: IMPALA-11349: Bump CDP_BUILD_NUMBER to 27992803 .. Patch Set 2: Verified+1 Code-Review+2 Thanks for the review Csaba. Carrying over +2 and adding verified +1 because the child verify job succeeded: https://gerrit.cloudera.org/#/c/18618/ -- To view, visit http://gerrit.cloudera.org:8080/18613 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9fcef254a5f845540e09e0d8d2111f305fb2fea2 Gerrit-Change-Number: 18613 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Mon, 20 Jun 2022 13:48:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11349: Bump CDP BUILD NUMBER to 27992803
Tamas Mate has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18613 ) Change subject: IMPALA-11349: Bump CDP_BUILD_NUMBER to 27992803 .. IMPALA-11349: Bump CDP_BUILD_NUMBER to 27992803 This commit bumps the CDP Build number to 27992803 and versions to 7.2.16.0-77. The new version includes https://github.com/apache/iceberg/pull/2925, an Iceberg library improvement to support serializable and snapshot isolation for ReplacePartitions. This is a prereuquisite to safely execute INSERT OVERWRITE entire partitions. Testing: - Ran a build and verified that the downloads succeed. Change-Id: I9fcef254a5f845540e09e0d8d2111f305fb2fea2 Reviewed-on: http://gerrit.cloudera.org:8080/18613 Reviewed-by: Tamas Mate Tested-by: Tamas Mate --- M bin/impala-config.sh 1 file changed, 12 insertions(+), 12 deletions(-) Approvals: Tamas Mate: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/18613 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I9fcef254a5f845540e09e0d8d2111f305fb2fea2 Gerrit-Change-Number: 18613 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-11356: Fix test sfs.py to account for new Hive behavior
Tamas Mate has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18618 ) Change subject: IMPALA-11356: Fix test_sfs.py to account for new Hive behavior .. IMPALA-11356: Fix test_sfs.py to account for new Hive behavior Hive changed behavior in HIVE-24920 to allow external SFS tables to point to locations in managed tables. test_sfs.py had a test case for the previous failure, so it broke with this change. This changes test_sfs.py to verify the new behavior where an external SFS table can point to a managed location. Testing: - Ran test_sfs.py locally with recent GBN Change-Id: I426aacdba7afba3f3a747a4e02632d15e38c63c9 Reviewed-on: http://gerrit.cloudera.org:8080/18618 Reviewed-by: Csaba Ringhofer Tested-by: Impala Public Jenkins --- M testdata/workloads/functional-query/queries/QueryTest/sfs.test 1 file changed, 63 insertions(+), 5 deletions(-) Approvals: Csaba Ringhofer: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/18618 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I426aacdba7afba3f3a747a4e02632d15e38c63c9 Gerrit-Change-Number: 18618 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-11367: Fix some formatting bugs when DESCRIBE FORMATTED for Iceberg tables
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18634 ) Change subject: IMPALA-11367: Fix some formatting bugs when DESCRIBE FORMATTED for Iceberg tables .. IMPALA-11367: Fix some formatting bugs when DESCRIBE FORMATTED for Iceberg tables Fix some formatting bugs when DESCRIBE FORMATTED for Iceberg tables: - 'LINE_DELIM' is missing on '# Partition Transform Information' - The partition transform columns header should be 'col_name,transform_type,NULL' Testing: - Existing tests Change-Id: I991644cefb34decc843a5542b47eaec11d7b6e42 Reviewed-on: http://gerrit.cloudera.org:8080/18634 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/util/HiveMetadataFormatUtils.java M testdata/workloads/functional-query/queries/QueryTest/iceberg-create.test 2 files changed, 23 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/18634 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I991644cefb34decc843a5542b47eaec11d7b6e42 Gerrit-Change-Number: 18634 Gerrit-PatchSet: 6 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-11367: Fix some formatting bugs when DESCRIBE FORMATTED for Iceberg tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18634 ) Change subject: IMPALA-11367: Fix some formatting bugs when DESCRIBE FORMATTED for Iceberg tables .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/18634 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I991644cefb34decc843a5542b47eaec11d7b6e42 Gerrit-Change-Number: 18634 Gerrit-PatchSet: 5 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 20 Jun 2022 14:06:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11358: Fixed Kudu table's missing comment
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18627 ) Change subject: IMPALA-11358: Fixed Kudu table's missing comment .. Patch Set 8: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8249/ -- To view, visit http://gerrit.cloudera.org:8080/18627 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf66f8b4679b00da6693a27fed79b04e8f6afb55 Gerrit-Change-Number: 18627 Gerrit-PatchSet: 8 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 20 Jun 2022 14:15:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11341: Print error log files when data-loading fails
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/18597 ) Change subject: IMPALA-11341: Print error log files when data-loading fails .. Patch Set 1: Code-Review+1 Looks good to me. I don't mind increasing the line count from 50 to 100 though. -- To view, visit http://gerrit.cloudera.org:8080/18597 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I771b4b74e00d89a3cc85f26b8b1da66c3dd66304 Gerrit-Change-Number: 18597 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Mon, 20 Jun 2022 14:42:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11358: Fixed Kudu table's missing comment
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18627 ) Change subject: IMPALA-11358: Fixed Kudu table's missing comment .. Patch Set 9: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18627 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf66f8b4679b00da6693a27fed79b04e8f6afb55 Gerrit-Change-Number: 18627 Gerrit-PatchSet: 9 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 20 Jun 2022 14:47:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11358: Fixed Kudu table's missing comment
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18627 ) Change subject: IMPALA-11358: Fixed Kudu table's missing comment .. Patch Set 9: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8252/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/18627 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf66f8b4679b00da6693a27fed79b04e8f6afb55 Gerrit-Change-Number: 18627 Gerrit-PatchSet: 9 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 20 Jun 2022 14:47:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11366: Fix impala-shell build failure if the build date contains Non-ASCII characters
Gergely Fürnstáhl has posted comments on this change. ( http://gerrit.cloudera.org:8080/18632 ) Change subject: IMPALA-11366: Fix impala-shell build failure if the build date contains Non-ASCII characters .. Patch Set 2: Code-Review+1 LGTM, tested with hungarian language -- To view, visit http://gerrit.cloudera.org:8080/18632 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I596121e08a52a4bc6d8668cf7e8b61b6c34eb4b9 Gerrit-Change-Number: 18632 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Mon, 20 Jun 2022 15:15:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11341: Print error log files when data-loading fails
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/18597 ) Change subject: IMPALA-11341: Print error log files when data-loading fails .. Patch Set 1: Code-Review+2 LGTM -- To view, visit http://gerrit.cloudera.org:8080/18597 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I771b4b74e00d89a3cc85f26b8b1da66c3dd66304 Gerrit-Change-Number: 18597 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Mon, 20 Jun 2022 15:21:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9726: Update impala-shell PyPI sidebar
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18575 ) Change subject: IMPALA-9726: Update impala-shell PyPI sidebar .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18575 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee12ff20499ca88b2acb56355a89cad288556c9e Gerrit-Change-Number: 18575 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 20 Jun 2022 16:29:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9726: Update impala-shell PyPI sidebar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18575 ) Change subject: IMPALA-9726: Update impala-shell PyPI sidebar .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8253/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/18575 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee12ff20499ca88b2acb56355a89cad288556c9e Gerrit-Change-Number: 18575 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 20 Jun 2022 16:46:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9726: Update impala-shell PyPI sidebar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18575 ) Change subject: IMPALA-9726: Update impala-shell PyPI sidebar .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18575 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee12ff20499ca88b2acb56355a89cad288556c9e Gerrit-Change-Number: 18575 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 20 Jun 2022 16:46:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9718: Delete pkg resources from impala-shell
Hello Wenzhe Zhou, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18585 to look at the new patch set (#5). Change subject: IMPALA-9718: Delete pkg_resources from impala-shell .. IMPALA-9718: Delete pkg_resources from impala-shell This reverts commit 0a0001e1a85462c81c9c4617a2e864c98913f229 to re-apply IMPALA-9718. CentOS 7 has pkg_resources available in its latest Python 2.7 release; we may need to install python-setuptools if not present in test environments. Actually uses PYTHON_EGG_CACHE set in the impala-shell script. This has an extra benefit of avoiding putting PYTHON_EGG_CACHE in the Jenkins user environment, which is group-writable and causes > UserWarning: /var/lib/jenkins/.python-eggs is writable by group/others and vulnerable to attack when used with get_resource_filename. Change-Id: I8e443d78671d8afab70d784664e71a70ccfcd587 --- M LICENSE.txt M shell/impala-shell M shell/make_shell_tarball.sh D shell/pkg_resources.py 4 files changed, 1 insertion(+), 2,703 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/18585/5 -- To view, visit http://gerrit.cloudera.org:8080/18585 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8e443d78671d8afab70d784664e71a70ccfcd587 Gerrit-Change-Number: 18585 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-9718: Delete pkg resources from impala-shell
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/18585 ) Change subject: IMPALA-9718: Delete pkg_resources from impala-shell .. Patch Set 5: New build with PYTHON_EGG_CACHE correctly exported in impala-shell: https://master-03.jenkins.cloudera.com/job/impala-private-parameterized/1090/ -- To view, visit http://gerrit.cloudera.org:8080/18585 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8e443d78671d8afab70d784664e71a70ccfcd587 Gerrit-Change-Number: 18585 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 20 Jun 2022 17:09:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11366: Fix impala-shell build failure if the build date contains Non-ASCII characters
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18632 ) Change subject: IMPALA-11366: Fix impala-shell build failure if the build date contains Non-ASCII characters .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/18632 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I596121e08a52a4bc6d8668cf7e8b61b6c34eb4b9 Gerrit-Change-Number: 18632 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Mon, 20 Jun 2022 17:15:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11366: Fix impala-shell build failure if the build date contains Non-ASCII characters
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18632 ) Change subject: IMPALA-11366: Fix impala-shell build failure if the build date contains Non-ASCII characters .. IMPALA-11366: Fix impala-shell build failure if the build date contains Non-ASCII characters The version string is generated in bin/save-version.sh where the BUILD_TIME is the result of `date` command. It could contain Non-ASCII characters depending on the locale. The date string is then used in generating impala_build_version.py. So we need to explicitly set the script's encoding to utf-8. Change-Id: I596121e08a52a4bc6d8668cf7e8b61b6c34eb4b9 Reviewed-on: http://gerrit.cloudera.org:8080/18632 Reviewed-by: Impala Public Jenkins Reviewed-by: Gergely Fürnstáhl Tested-by: Impala Public Jenkins --- M shell/make_shell_tarball.sh 1 file changed, 1 insertion(+), 0 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified Gergely Fürnstáhl: Looks good to me, but someone else must approve -- To view, visit http://gerrit.cloudera.org:8080/18632 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I596121e08a52a4bc6d8668cf7e8b61b6c34eb4b9 Gerrit-Change-Number: 18632 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith
[Impala-ASF-CR] IMPALA-11233: Unset all query option
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18430 ) Change subject: IMPALA-11233: Unset all query option .. Patch Set 14: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8251/ -- To view, visit http://gerrit.cloudera.org:8080/18430 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iabf23622daab733ddab20dd3ca73af6c9bd5c250 Gerrit-Change-Number: 18430 Gerrit-PatchSet: 14 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jian Zhang Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Xiaoqing Gao Gerrit-Comment-Date: Mon, 20 Jun 2022 17:20:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9718: Delete pkg resources from impala-shell
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18585 ) Change subject: IMPALA-9718: Delete pkg_resources from impala-shell .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10813/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18585 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8e443d78671d8afab70d784664e71a70ccfcd587 Gerrit-Change-Number: 18585 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 20 Jun 2022 17:26:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10316: Increase Yarn minimum container size for dataload
Laszlo Gaal (Cloudera) has posted comments on this change. ( http://gerrit.cloudera.org:8080/18630 ) Change subject: IMPALA-10316: Increase Yarn minimum container size for dataload .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18630 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I77e7c9e9fa3491c6e5652351869d3a4410bbb7b8 Gerrit-Change-Number: 18630 Gerrit-PatchSet: 4 Gerrit-Owner: Laszlo Gaal (Cloudera) Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal (Cloudera) Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Mon, 20 Jun 2022 18:44:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10316: Increase Yarn minimum container size for dataload
Laszlo Gaal (Cloudera) has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18630 ) Change subject: IMPALA-10316: Increase Yarn minimum container size for dataload .. IMPALA-10316: Increase Yarn minimum container size for dataload This is an attempt to get rod of IMPALA-10669 and friends, crashing Tez containers during the loading of nested ORC data. The usual error message logged for these failures is: Container [pid=11530,containerID=container_1618776748992_0039_01_03] is running 2785280B beyond the 'PHYSICAL' memory limit. Current usage: 1.0 GB of 1 GB physical memory used; 2.6 GB of 2.1 GB virtual memory used. Killing container. https://stackoverflow.com/a/43827548/143681 explains that the tunable setting 'yarn.scheduler.minimum-allocation-mb' in yarn-site.xml sets both the minimum memory size and the memory size increment for Yarn containers This patch is an attempt to work around the failure by forcibly setting a minimum size for the Yarn containers used in dataload that is significantly larger than the 1 GB size reported in the failure messages. Tested by running the dataload phase successfully on the following platform combinations: - Ubuntu 16.04, m6i.8xlarge (128 GB RAM, Docker) - Ubuntu 16.04, m5.12xlarge (192 GB RAM, Docker) - Centos 7.4, m5.4xlarge (64 GB RAM) - Centos 7.4, r5.4xlarge (128 GB RAM) - Ubuntu 16.04, m6i.4xlarge (64 GB RAM) Change-Id: I77e7c9e9fa3491c6e5652351869d3a4410bbb7b8 Reviewed-on: http://gerrit.cloudera.org:8080/18630 Tested-by: Impala Public Jenkins Reviewed-by: Michael Smith Reviewed-by: Laszlo Gaal (Cloudera) --- M testdata/cluster/node_templates/common/etc/hadoop/conf/yarn-site.xml.py 1 file changed, 4 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Verified Michael Smith: Looks good to me, but someone else must approve Laszlo Gaal (Cloudera): Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/18630 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I77e7c9e9fa3491c6e5652351869d3a4410bbb7b8 Gerrit-Change-Number: 18630 Gerrit-PatchSet: 5 Gerrit-Owner: Laszlo Gaal (Cloudera) Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal (Cloudera) Gerrit-Reviewer: Michael Smith
[Impala-ASF-CR] IMPALA-10316: Increase tez container mem limit in load nested.py
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/18553 ) Change subject: IMPALA-10316: Increase tez container mem limit in load_nested.py .. Patch Set 2: Code-Review-1 (1 comment) Redundant with https://gerrit.cloudera.org/c/18630/ merged. http://gerrit.cloudera.org:8080/#/c/18553/2/testdata/bin/load_nested.py File testdata/bin/load_nested.py: http://gerrit.cloudera.org:8080/#/c/18553/2/testdata/bin/load_nested.py@308 PS2, Line 308: SET hive.tez.container.size=2048; > Ah, this only applies to certain loads. And the bug report is specifically Ok, so this does kick in. One outcome is that the heap increases to `-Xmx1638m`. It likely will avoid killing containers because that 20% memory for system and JVM overhead is now larger as well. My change SET tez.container.max.java.heap.fraction=0.75; feels a bit more targeted. The increased heap doesn't show any improvement in load time on my system (lowering fraction took 55s to load, increasing container size took 56s to load). -- To view, visit http://gerrit.cloudera.org:8080/18553 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id5589ed39f9073e485d0044f2d07fa506ac25574 Gerrit-Change-Number: 18553 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Comment-Date: Mon, 20 Jun 2022 18:45:13 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10316: refine tez heap allocation
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/18612 ) Change subject: IMPALA-10316: refine tez heap allocation .. Patch Set 5: Closing in favor of https://gerrit.cloudera.org/c/18630/. -- To view, visit http://gerrit.cloudera.org:8080/18612 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0a35e6c9149ee4c84aee4123d2c47a378591b621 Gerrit-Change-Number: 18612 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Mon, 20 Jun 2022 18:45:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10316: refine tez heap allocation
Michael Smith has abandoned this change. ( http://gerrit.cloudera.org:8080/18612 ) Change subject: IMPALA-10316: refine tez heap allocation .. Abandoned Closing in favor of https://gerrit.cloudera.org/c/18630/. -- To view, visit http://gerrit.cloudera.org:8080/18612 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I0a35e6c9149ee4c84aee4123d2c47a378591b621 Gerrit-Change-Number: 18612 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-11358: Fixed Kudu table's missing comment
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18627 ) Change subject: IMPALA-11358: Fixed Kudu table's missing comment .. Patch Set 9: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/18627 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf66f8b4679b00da6693a27fed79b04e8f6afb55 Gerrit-Change-Number: 18627 Gerrit-PatchSet: 9 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 20 Jun 2022 19:38:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11358: Fixed Kudu table's missing comment
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18627 ) Change subject: IMPALA-11358: Fixed Kudu table's missing comment .. IMPALA-11358: Fixed Kudu table's missing comment If Kudu-HMS integration is enabled, Kudu creates the table in HMS too, which was missing the comment field. Added the code to forward the comment field to Kudu during creation. Testing: Added a test to verify the comment is present when the intergration is enabled. Reenabled several kudu tests as IMPALA-8751 (and follow ups) fixed the hive3 notification incompatibility. Change-Id: Idf66f8b4679b00da6693a27fed79b04e8f6afb55 Reviewed-on: http://gerrit.cloudera.org:8080/18627 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M tests/common/skip.py M tests/custom_cluster/test_kudu.py 3 files changed, 12 insertions(+), 14 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/18627 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Idf66f8b4679b00da6693a27fed79b04e8f6afb55 Gerrit-Change-Number: 18627 Gerrit-PatchSet: 10 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-11369: Separate thrift compiler for different component
Joe McDonnell has posted comments on this change. (
http://gerrit.cloudera.org:8080/18636 )
Change subject: IMPALA-11369: Separate thrift compiler for different component
..
Patch Set 4:
(1 comment)
This is looking good to me. I had one small comment.
http://gerrit.cloudera.org:8080/#/c/18636/4/common/thrift/CMakeLists.txt
File common/thrift/CMakeLists.txt:
http://gerrit.cloudera.org:8080/#/c/18636/4/common/thrift/CMakeLists.txt@131
PS4, Line 131: set(THRIFT_INCLUDE_DIR_OPTION -I ${THRIFT_CPP_CONTRIB_DIR} -I
$ENV{HIVE_METASTORE_THRIFT_DIR}
: -I ${HIVE_THRIFT_SOURCE_DIR})
This is used for all three languages (see CPP_ARGS, JAVA_FE_ARGS,
JAVA_EXT_DS_ARGS, PYTHON_ARGS, etc). We'll need to split it out for the
different languages, so let's go ahead and do that.
--
To view, visit http://gerrit.cloudera.org:8080/18636
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I56479dc69b79024d1a4d09211bbe88a61fa0c6a4
Gerrit-Change-Number: 18636
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Riza Suminto
Gerrit-Comment-Date: Mon, 20 Jun 2022 21:15:23 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9726: Update impala-shell PyPI sidebar
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18575 ) Change subject: IMPALA-9726: Update impala-shell PyPI sidebar .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/18575 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee12ff20499ca88b2acb56355a89cad288556c9e Gerrit-Change-Number: 18575 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Mon, 20 Jun 2022 21:32:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9726: Update impala-shell PyPI sidebar
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18575 ) Change subject: IMPALA-9726: Update impala-shell PyPI sidebar .. IMPALA-9726: Update impala-shell PyPI sidebar Updates impala-shell's PyPI sidebar classifiers to reflect support for Python 2.6+. Change-Id: Iee12ff20499ca88b2acb56355a89cad288556c9e Reviewed-on: http://gerrit.cloudera.org:8080/18575 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M shell/packaging/setup.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/18575 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iee12ff20499ca88b2acb56355a89cad288556c9e Gerrit-Change-Number: 18575 Gerrit-PatchSet: 3 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-11369: Separate thrift compiler for different component
Hello Joe McDonnell, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18636 to look at the new patch set (#5). Change subject: IMPALA-11369: Separate thrift compiler for different component .. IMPALA-11369: Separate thrift compiler for different component Impala used to have one thrift compiler version to compile C++, Java, and Python code. Most Thrift serialization/deserialization between minor versions are compatible with each other. So it is possible to have different thrift compiler versions for different target codes. It is beneficial to do so because it will allow Impala to upgrade separate components independently. This patch implements the infrastructure change required to do so. It replace most of the 'THRIFT_*' environment variable and CMake variable with 'THRFIT_CPP_*', 'THRFIT_JAVA_*', and 'THRFIT_PY_*' to compile C++, Java, and Python code accordingly. All three still refer to the same thrift version (thrift-0.11.0-p5). Testing: - Build Impala and pass core tests. Change-Id: I56479dc69b79024d1a4d09211bbe88a61fa0c6a4 --- M CMakeLists.txt M README-build.md M be/CMakeLists.txt M bin/bootstrap_toolchain.py M bin/impala-config.sh M bin/impala-shell.sh M bin/set-pythonpath.sh D cmake_modules/FindThrift.cmake A cmake_modules/FindThriftCpp.cmake A cmake_modules/FindThriftJava.cmake A cmake_modules/FindThriftPython.cmake M common/thrift/CMakeLists.txt M java/pom.xml M shell/make_shell_tarball.sh M shell/packaging/make_python_package.sh 15 files changed, 312 insertions(+), 139 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/36/18636/5 -- To view, visit http://gerrit.cloudera.org:8080/18636 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I56479dc69b79024d1a4d09211bbe88a61fa0c6a4 Gerrit-Change-Number: 18636 Gerrit-PatchSet: 5 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Riza Suminto
[Impala-ASF-CR] IMPALA-11369: Separate thrift compiler for different component
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/18636 )
Change subject: IMPALA-11369: Separate thrift compiler for different component
..
Patch Set 5:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/18636/5/bin/impala-config.sh
File bin/impala-config.sh:
http://gerrit.cloudera.org:8080/#/c/18636/5/bin/impala-config.sh@762
PS5, Line 762: export
THRIFT_CPP_HOME="${IMPALA_TOOLCHAIN_PACKAGES_HOME}/thrift-${IMPALA_THRIFT_CPP_VERSION}"
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/18636/5/bin/impala-config.sh@763
PS5, Line 763: export
THRIFT_JAVA_HOME="${IMPALA_TOOLCHAIN_PACKAGES_HOME}/thrift-${IMPALA_THRIFT_JAVA_VERSION}"
line too long (96 > 90)
http://gerrit.cloudera.org:8080/#/c/18636/5/bin/impala-config.sh@764
PS5, Line 764: export
THRIFT_PY_HOME="${IMPALA_TOOLCHAIN_PACKAGES_HOME}/thrift-${IMPALA_THRIFT_PY_VERSION}"
line too long (92 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/18636
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I56479dc69b79024d1a4d09211bbe88a61fa0c6a4
Gerrit-Change-Number: 18636
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Riza Suminto
Gerrit-Comment-Date: Mon, 20 Jun 2022 21:34:17 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11369: Separate thrift compiler for different component
Hello Joe McDonnell, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18636 to look at the new patch set (#6). Change subject: IMPALA-11369: Separate thrift compiler for different component .. IMPALA-11369: Separate thrift compiler for different component Impala used to have one thrift compiler version to compile C++, Java, and Python code. Most Thrift serialization/deserialization between minor versions are compatible with each other. So it is possible to have different thrift compiler versions for different target codes. It is beneficial to do so because it will allow Impala to upgrade separate components independently. This patch implements the infrastructure change required to do so. It replace most of the 'THRIFT_*' environment variable and CMake variable with 'THRFIT_CPP_*', 'THRFIT_JAVA_*', and 'THRFIT_PY_*' to compile C++, Java, and Python code accordingly. All three still refer to the same thrift version (thrift-0.11.0-p5). Testing: - Build Impala and pass core tests. Change-Id: I56479dc69b79024d1a4d09211bbe88a61fa0c6a4 --- M CMakeLists.txt M README-build.md M be/CMakeLists.txt M bin/bootstrap_toolchain.py M bin/impala-config.sh M bin/impala-shell.sh M bin/set-pythonpath.sh D cmake_modules/FindThrift.cmake A cmake_modules/FindThriftCpp.cmake A cmake_modules/FindThriftJava.cmake A cmake_modules/FindThriftPython.cmake M common/thrift/CMakeLists.txt M java/pom.xml M shell/make_shell_tarball.sh M shell/packaging/make_python_package.sh 15 files changed, 314 insertions(+), 139 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/36/18636/6 -- To view, visit http://gerrit.cloudera.org:8080/18636 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I56479dc69b79024d1a4d09211bbe88a61fa0c6a4 Gerrit-Change-Number: 18636 Gerrit-PatchSet: 6 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Riza Suminto
[Impala-ASF-CR] IMPALA-11369: Separate thrift compiler for different component
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/18636 )
Change subject: IMPALA-11369: Separate thrift compiler for different component
..
Patch Set 6:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/18636/6/bin/impala-config.sh
File bin/impala-config.sh:
http://gerrit.cloudera.org:8080/#/c/18636/6/bin/impala-config.sh@762
PS6, Line 762: export
THRIFT_CPP_HOME="${IMPALA_TOOLCHAIN_PACKAGES_HOME}/thrift-${IMPALA_THRIFT_CPP_VERSION}"
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/18636/6/bin/impala-config.sh@763
PS6, Line 763: export
THRIFT_JAVA_HOME="${IMPALA_TOOLCHAIN_PACKAGES_HOME}/thrift-${IMPALA_THRIFT_JAVA_VERSION}"
line too long (96 > 90)
http://gerrit.cloudera.org:8080/#/c/18636/6/bin/impala-config.sh@764
PS6, Line 764: export
THRIFT_PY_HOME="${IMPALA_TOOLCHAIN_PACKAGES_HOME}/thrift-${IMPALA_THRIFT_PY_VERSION}"
line too long (92 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/18636
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I56479dc69b79024d1a4d09211bbe88a61fa0c6a4
Gerrit-Change-Number: 18636
Gerrit-PatchSet: 6
Gerrit-Owner: Riza Suminto
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Riza Suminto
Gerrit-Comment-Date: Mon, 20 Jun 2022 21:37:51 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10791 Add batch reading for remote temporary files
Michael Smith has posted comments on this change. (
http://gerrit.cloudera.org:8080/17979 )
Change subject: IMPALA-10791 Add batch reading for remote temporary files
..
Patch Set 12:
(19 comments)
A mix of questions and suggestions. I didn't see anything clearly wrong.
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/disk-file.h
File be/src/runtime/io/disk-file.h:
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/disk-file.h@55
PS12, Line 55: class MemBlock {
This seems like a fairly leaky abstraction. It mostly exists to handle the
lifecycle around data_, but also relies on external operations to do lots of
the actual management. Not sure I have specific improvements to suggest however.
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/disk-file.h@60
PS12, Line 60: DCHECK_EQ(static_cast(status_),
static_cast(MemBlockStatus::DISABLED));
Can we also verify no memory leaks by checking `data_`?
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/disk-file.h@220
PS12, Line 220: std::unique_ptr page_cnts_per_block_;
This seems a little funny, but I think I see why it's necessary. A const vector
wouldn't let you update element values, and a non-const vector could
potentially be resized. This is the only way to get a runtime fixed-size array.
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/disk-file.h@386
PS12, Line 386: bool DoReadFromReadBuffer(
This function doesn't take any action, so I'm not really sure why it's called
"DoRead". I'd opt for Can, Should, or Must (depending on expectations here).
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/disk-file.h@446
PS12, Line 446: // Helper function to check the status of a read buffe block.
nit: buffe -> buffer
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/disk-file.h@458
PS12, Line 458: // Helper function to delete the read buffe block.
nit: buffe -> buffer
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/disk-file.h@516
PS12, Line 516: /// The lock also protects the memory blocks from
destruction, if the disk file has.
"if the disk file has" seems like an incomplete sentence.
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/disk-file.cc
File be/src/runtime/io/disk-file.cc:
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/disk-file.cc@99
PS12, Line 99: void MemBlock::Delete(bool* reserved, bool* allocated) {
Some unit tests around MemBlock transitions might be useful.
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/disk-file.cc@116
PS12, Line 116: SetStatusLocked(lock, MemBlockStatus::DISABLED);
It would be nice to have a DCHECK(data_ == nullptr) here. It would be true
immediately after freeing ALLOC memory, or in other states (because no memory
should have been allocated).
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/disk-io-mgr.cc
File be/src/runtime/io/disk-io-mgr.cc:
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/disk-io-mgr.cc@463
PS12, Line 463: // If there was an error during reading, keep the old
status.
Might be helpful to log errors that we can't return.
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/request-ranges.h
File be/src/runtime/io/request-ranges.h:
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/request-ranges.h@152
PS12, Line 152: RequestRange(RequestType::type request_type, int disk_id =
-1, int64_t offset = -1)
How does an offset of -1 differ from 0?
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/scan-range.cc
File be/src/runtime/io/scan-range.cc:
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/scan-range.cc@156
PS12, Line 156: if (!use_mem_buffer) {
Since we asserted above that use_local_buff implies !use_mem_buffer, this check
is redundant.
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/io/scan-range.cc@291
PS12, Line 291: // 1. If it is the local buffer file is not
deleted(evicted) yet.
Change to "If the local buffer file ..."
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/tmp-file-mgr-test.cc
File be/src/runtime/tmp-file-mgr-test.cc:
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/tmp-file-mgr-test.cc@1835
PS12, Line 1835: int64_t file_size = 512 * 1024 * 1024;
Why was this changed?
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/tmp-file-mgr.cc
File be/src/runtime/tmp-file-mgr.cc:
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/tmp-file-mgr.cc@108
PS12, Line 108: "files. Only valid when the remote_batch_read is true.");
"Only valid when remote_batch_read is true." seems more consistent with how we
talk about other flags.
http://gerrit.cloudera.org:8080/#/c/17979/12/be/src/runtime/tmp-file-mgr.cc@117
PS12, Line 117: "Set if the system uses batch reading for the
[Impala-ASF-CR] IMPALA-11369: Separate thrift compiler for different component
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18636 ) Change subject: IMPALA-11369: Separate thrift compiler for different component .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10814/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18636 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I56479dc69b79024d1a4d09211bbe88a61fa0c6a4 Gerrit-Change-Number: 18636 Gerrit-PatchSet: 5 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Mon, 20 Jun 2022 21:53:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11369: Separate thrift compiler for different component
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18636 ) Change subject: IMPALA-11369: Separate thrift compiler for different component .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10815/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18636 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I56479dc69b79024d1a4d09211bbe88a61fa0c6a4 Gerrit-Change-Number: 18636 Gerrit-PatchSet: 6 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Mon, 20 Jun 2022 21:57:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11369: Separate thrift compiler for different component
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/18636 ) Change subject: IMPALA-11369: Separate thrift compiler for different component .. Patch Set 6: Code-Review+2 This looks good to me -- To view, visit http://gerrit.cloudera.org:8080/18636 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I56479dc69b79024d1a4d09211bbe88a61fa0c6a4 Gerrit-Change-Number: 18636 Gerrit-PatchSet: 6 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Mon, 20 Jun 2022 22:01:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11369: Separate thrift compiler for different component
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18636 ) Change subject: IMPALA-11369: Separate thrift compiler for different component .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18636 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I56479dc69b79024d1a4d09211bbe88a61fa0c6a4 Gerrit-Change-Number: 18636 Gerrit-PatchSet: 7 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Mon, 20 Jun 2022 22:04:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11369: Separate thrift compiler for different component
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18636 ) Change subject: IMPALA-11369: Separate thrift compiler for different component .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8254/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/18636 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I56479dc69b79024d1a4d09211bbe88a61fa0c6a4 Gerrit-Change-Number: 18636 Gerrit-PatchSet: 7 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Mon, 20 Jun 2022 22:04:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9615: re2's max mem opt configurable via an Impala startup flag
Abhishek Rawat has posted comments on this change. (
http://gerrit.cloudera.org:8080/18602 )
Change subject: IMPALA-9615: re2's max_mem opt configurable via an Impala
startup flag
..
Patch Set 9:
(7 comments)
Overall looks good to me. I just have some minor comments and suggestions for
follow on tasks.
http://gerrit.cloudera.org:8080/#/c/18602/9//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18602/9//COMMIT_MSG@9
PS9, Line 9: Some regex patterns require more memory to be compiled and pattern
matched using
Would be good to wrap the commit message lines within 72 chars as mentioned
here: https://cwiki.apache.org/confluence/display/IMPALA/Contributing+to+Impala
http://gerrit.cloudera.org:8080/#/c/18602/9//COMMIT_MSG@19
PS9, Line 19: The testcase uses a long regex pattern to use up all the memory
in the
Maybe, a good idea to include testing details in a 'Testing' section. Here is a
recent commit which is a good example to follow:
https://github.com/apache/impala/commit/d12d4c9b077392f58701af2e343dd927fe516846
http://gerrit.cloudera.org:8080/#/c/18602/9/be/src/exprs/like-predicate.cc
File be/src/exprs/like-predicate.cc:
http://gerrit.cloudera.org:8080/#/c/18602/9/be/src/exprs/like-predicate.cc@104
PS9, Line 104: StringFunctions::SetRE2MemOpt(&opts);
Would be interesting to run some tests to measure the perf impact of this patch.
http://gerrit.cloudera.org:8080/#/c/18602/9/be/src/exprs/string-functions-ir.cc
File be/src/exprs/string-functions-ir.cc:
http://gerrit.cloudera.org:8080/#/c/18602/9/be/src/exprs/string-functions-ir.cc@46
PS9, Line 46: DECLARE_string(re2_mem_limit);
Is this being used somewhere?
http://gerrit.cloudera.org:8080/#/c/18602/9/be/src/exprs/string-functions-ir.cc@934
PS9, Line 934: void StringFunctions::SetRE2MemLimit(int64_t re2_mem_limit) {
Unfortunately, this is another example of untracked memory usage at runtime. We
could have multiple instances of re2 (at least one per fragment) and each could
use the configured amount of memory. Unfortunately, the planner doesn't know of
any such memory requirement from the LIKE and other predicates using
StringFunctions and so doesn't quite predict the real memory requirement. Would
be good to include the memory requirement for re2 in the planner estimates.
This probably should be a follow on JIRA.
http://gerrit.cloudera.org:8080/#/c/18602/9/tests/custom_cluster/test_re2_max_mem.py
File tests/custom_cluster/test_re2_max_mem.py:
http://gerrit.cloudera.org:8080/#/c/18602/9/tests/custom_cluster/test_re2_max_mem.py@30
PS9, Line 30: test to see given an amount of memory (in Bytes) does
the re2 print an error for
nit: "for for DFA" -> "for DFA"
http://gerrit.cloudera.org:8080/#/c/18602/9/tests/custom_cluster/test_re2_max_mem.py@43
PS9, Line 43: if expect_fail:
If 'expect_fail' is set then we seem to be ignoring 'dfa_out_of_mem' flag. This
is probably okay, but maybe something to add in the test comments.
--
To view, visit http://gerrit.cloudera.org:8080/18602
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idf28d2f7217b1322ab8fdfb2c02fff0608078571
Gerrit-Change-Number: 18602
Gerrit-PatchSet: 9
Gerrit-Owner: Omid Shahidi
Gerrit-Reviewer: Abhishek Rawat
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Kurt Deschler
Gerrit-Reviewer: Omid Shahidi
Gerrit-Reviewer: Riza Suminto
Gerrit-Comment-Date: Mon, 20 Jun 2022 22:55:29 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11064 Optimizing Temporary File Structure for Batch Reading
Michael Smith has posted comments on this change. (
http://gerrit.cloudera.org:8080/18219 )
Change subject: IMPALA-11064 Optimizing Temporary File Structure for Batch
Reading
..
Patch Set 5:
(18 comments)
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/exec/partitioned-hash-join-builder.h
File be/src/exec/partitioned-hash-join-builder.h:
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/exec/partitioned-hash-join-builder.h@80
PS5, Line 80: /// left shift 16 bits for the later specified partition id.
PartitionId is an int, which can be 32-bits. Should we left shift 32 bits here?
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/exec/partitioned-hash-join-builder.h@82
PS5, Line 82: static uint64_t GenerateBaseSpillId(int64_t level, uint64_t
default_id) {
Can you add a small unit test for this?
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/buffered-tuple-stream.h
File be/src/runtime/buffered-tuple-stream.h:
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/buffered-tuple-stream.h@422
PS5, Line 422: void ResetSpillId() { spill_id_ = 0; }
This seems to be unused. When would it be needed?
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/tmp-file-mgr-internal.h
File be/src/runtime/tmp-file-mgr-internal.h:
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/tmp-file-mgr-internal.h@847
PS5, Line 847: /// Shut down the pool before destruction.
Is it a loop or a pool? Or both? Mixing them makes it seem like there's a typo.
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/tmp-file-mgr.h
File be/src/runtime/tmp-file-mgr.h:
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/tmp-file-mgr.h@617
PS5, Line 617: uint64_t EnqueueTmpBlockBySpillId(uint64_t spill_id, TmpBlock*
tmp_block) {
This doesn't require any locking?
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/tmp-file-mgr.cc
File be/src/runtime/tmp-file-mgr.cc:
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/tmp-file-mgr.cc@130
PS5, Line 130: "The size should be power of 2 and integer times of the
block size. ");
Maybe "The size should be a power of 2, and an integer multiple of the block
size."
Does it really need to be a power of 2? Or does it just need to be an integer
multiple of the block size? Would remote_batch_read_max_block_size_level=3,
remote_batch_read_tmp_file_size=24M be fine?
Also mixing level and MB entries like this seems confusing and unnecessary.
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/tmp-file-mgr.cc@710
PS5, Line 710: // Chcek the unuploaded_files_accessing_ vector to see if the
access time of any file
nit: Chcek -> Check
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/tmp-file-mgr.cc@712
PS5, Line 712: while (true) {
Why isn't this
while(it != unuploaded_files_accessing_.end()) {
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/tmp-file-mgr.cc@721
PS5, Line 721: if (!tmp_file->TrySetFullBlocks()) {
This could all be
if (tmp_file->TrySetFullBlocks()) tmp_file->SetFullBatchReadFileHelper(true
/*is_force*/);
it = unuploaded_files_accessing_.erase(it);
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/tmp-file-mgr.cc@746
PS5, Line 746: if (!tmp_file->TrySetFullBlocks()) {
If you change the previous loop, I'd also invert this conditional.
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/tmp-file-mgr.cc@764
PS5, Line 764: // remote_batch_read_max_block_size_level. For example, it the
max level is 3,
nit: it -> if
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/tmp-file-mgr.cc@798
PS5, Line 798: >> 1;
Is the bit manipulation really worth it over multiplication/division? Seems
less clear.
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/tmp-file-mgr.cc@1343
PS5, Line 1343: // Caculate the actual content bytes and file size for the
batch reading file.
nit: Caculate -> Calculate
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/tmp-file-mgr.cc@1639
PS5, Line 1639: // Prefetch a block with spill id minus one if it is the
end of the current spill id.
Why is it minus one instead of plus one?
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/runtime/tmp-file-mgr.cc@1933
PS5, Line 1933: //
tmp_file_mgr_->tmp_dirs_remote_ctrl_.tmp_file_pool_->IsAnyWaiting();
Why is this code here but commented out?
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/service/query-options.cc
File be/src/service/query-options.cc:
http://gerrit.cloudera.org:8080/#/c/18219/5/be/src/service/query-options.cc@1259
PS5, Line 1259: if (result != StringParser::PARSE_SUCCESS || level < -1
|| level > 12) {
Why 12 instead of 16 (or 32)?
http://gerrit.cloudera.org:8080/#/c/18219/5/common/thrift/Query.thrift
File common/thrift/Query.thrift:
http://gerrit.cloudera.org:8080/#/c/18219/5/common/thrift/Query.thrift@598
PS5, Line 598
[Impala-ASF-CR] IMPALA-10316: Increase tez container mem limit in load nested.py
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18553 ) Change subject: IMPALA-10316: Increase tez container mem limit in load_nested.py .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/18553/2/testdata/bin/load_nested.py File testdata/bin/load_nested.py: http://gerrit.cloudera.org:8080/#/c/18553/2/testdata/bin/load_nested.py@308 PS2, Line 308: SET hive.tez.container.size=2048; > Ok, so this does kick in. One outcome is that the heap increases to `-Xmx16 Thanks for the experiments! I'm not sure if reducing the java heap fraction will lead to hitting OOM inside the JVM more likely, since 0.75GB is really a small heap. On the other hand, if the Tez container uses offheap memory (I have no ideas about this), reduing the heap size might not help. The GVO job runs on instances with 64GB RAM and queries in the load_nested.py are executed seriesly. I think bumping the memory limit in this script is better in resource usage. Anyway, let's see whether https://gerrit.cloudera.org/c/18630/ is enough to resolve the issue. -- To view, visit http://gerrit.cloudera.org:8080/18553 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id5589ed39f9073e485d0044f2d07fa506ac25574 Gerrit-Change-Number: 18553 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 21 Jun 2022 02:06:47 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10316: Increase tez container mem limit in load nested.py
Quanlong Huang has abandoned this change. ( http://gerrit.cloudera.org:8080/18553 ) Change subject: IMPALA-10316: Increase tez container mem limit in load_nested.py .. Abandoned Abandon for https://gerrit.cloudera.org/c/18630/ -- To view, visit http://gerrit.cloudera.org:8080/18553 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: Id5589ed39f9073e485d0044f2d07fa506ac25574 Gerrit-Change-Number: 18553 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-11341: Print error log files when data-loading fails
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18597 ) Change subject: IMPALA-11341: Print error log files when data-loading fails .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8255/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/18597 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I771b4b74e00d89a3cc85f26b8b1da66c3dd66304 Gerrit-Change-Number: 18597 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Tue, 21 Jun 2022 02:13:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11341: Print error log files when data-loading fails
Hello Andrew Sherman, Riza Suminto, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18597 to look at the new patch set (#2). Change subject: IMPALA-11341: Print error log files when data-loading fails .. IMPALA-11341: Print error log files when data-loading fails run-step will print the last 50 lines of the log file when the step fails. However, the data-loading log file just shows the log file names of the failed sql files. We still need to manually look into them for more details. This patch extends run-step to also print the last 100 lines of such log files. Tested locally by stopping impala cluster to mimic an error state and run these in my terminal: source testdata/bin/run-step.sh export LOG_DIR=tmp run-step "Loading functional-query data" load-functional-query.log bin/load-data.py -e exhaustive -w functional-query Change-Id: I771b4b74e00d89a3cc85f26b8b1da66c3dd66304 --- M testdata/bin/run-step.sh 1 file changed, 11 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/97/18597/2 -- To view, visit http://gerrit.cloudera.org:8080/18597 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I771b4b74e00d89a3cc85f26b8b1da66c3dd66304 Gerrit-Change-Number: 18597 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Riza Suminto
[Impala-ASF-CR] IMPALA-11341: Print error log files when data-loading fails
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18597 ) Change subject: IMPALA-11341: Print error log files when data-loading fails .. Patch Set 2: Code-Review+2 Sure, change 50 to 100. Carry the +2. -- To view, visit http://gerrit.cloudera.org:8080/18597 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I771b4b74e00d89a3cc85f26b8b1da66c3dd66304 Gerrit-Change-Number: 18597 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Tue, 21 Jun 2022 02:12:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11341: Print error log files when data-loading fails
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18597 ) Change subject: IMPALA-11341: Print error log files when data-loading fails .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10816/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18597 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I771b4b74e00d89a3cc85f26b8b1da66c3dd66304 Gerrit-Change-Number: 18597 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Tue, 21 Jun 2022 02:31:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11369: Separate thrift compiler for different component
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18636 ) Change subject: IMPALA-11369: Separate thrift compiler for different component .. Patch Set 7: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/18636 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I56479dc69b79024d1a4d09211bbe88a61fa0c6a4 Gerrit-Change-Number: 18636 Gerrit-PatchSet: 7 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Tue, 21 Jun 2022 02:40:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11369: Separate thrift compiler for different component
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18636 ) Change subject: IMPALA-11369: Separate thrift compiler for different component .. IMPALA-11369: Separate thrift compiler for different component Impala used to have one thrift compiler version to compile C++, Java, and Python code. Most Thrift serialization/deserialization between minor versions are compatible with each other. So it is possible to have different thrift compiler versions for different target codes. It is beneficial to do so because it will allow Impala to upgrade separate components independently. This patch implements the infrastructure change required to do so. It replace most of the 'THRIFT_*' environment variable and CMake variable with 'THRFIT_CPP_*', 'THRFIT_JAVA_*', and 'THRFIT_PY_*' to compile C++, Java, and Python code accordingly. All three still refer to the same thrift version (thrift-0.11.0-p5). Testing: - Build Impala and pass core tests. Change-Id: I56479dc69b79024d1a4d09211bbe88a61fa0c6a4 Reviewed-on: http://gerrit.cloudera.org:8080/18636 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M CMakeLists.txt M README-build.md M be/CMakeLists.txt M bin/bootstrap_toolchain.py M bin/impala-config.sh M bin/impala-shell.sh M bin/set-pythonpath.sh D cmake_modules/FindThrift.cmake A cmake_modules/FindThriftCpp.cmake A cmake_modules/FindThriftJava.cmake A cmake_modules/FindThriftPython.cmake M common/thrift/CMakeLists.txt M java/pom.xml M shell/make_shell_tarball.sh M shell/packaging/make_python_package.sh 15 files changed, 314 insertions(+), 139 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/18636 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I56479dc69b79024d1a4d09211bbe88a61fa0c6a4 Gerrit-Change-Number: 18636 Gerrit-PatchSet: 8 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Riza Suminto
[Impala-ASF-CR] IMPALA-11233: Unset all query option
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18430 ) Change subject: IMPALA-11233: Unset all query option .. Patch Set 14: (1 comment) We recently add tests on Python3. Please fix the test failures on this patch. http://gerrit.cloudera.org:8080/#/c/18430/14/tests/shell/test_shell_interactive.py File tests/shell/test_shell_interactive.py: http://gerrit.cloudera.org:8080/#/c/18430/14/tests/shell/test_shell_interactive.py@706 PS14, Line 706: assert "\tMT_DOP: " in result.stdout This fails in Python3: https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5879/testReport/shell.test_shell_interactive/TestImpalaShellInteractive/test_query_option_configuration_impala_shell__python3___strict_hs2_protocol__False___protocol__hs2___exec_optiontest_replan___1___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ -- To view, visit http://gerrit.cloudera.org:8080/18430 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iabf23622daab733ddab20dd3ca73af6c9bd5c250 Gerrit-Change-Number: 18430 Gerrit-PatchSet: 14 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jian Zhang Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Xiaoqing Gao Gerrit-Comment-Date: Tue, 21 Jun 2022 03:08:01 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions
Xiaoqing Gao has uploaded this change for review. (
http://gerrit.cloudera.org:8080/18641
Change subject: IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the
partitions
..
IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions
Currently, SHOW PARTITIONS on Iceberg tables only outputs the partition spec
which is not too useful.
Instead it should output the concrete partitions, number of files, number of
rows in each partitions,
E.g.:
SHOW PARTITIONS iceberg_partitioned;
'{"event_time_hour":"438296","action":"view"}',8,8
'{"event_time_hour":"438297","action":"click"}',6,6
'{"event_time_hour":"438298","action":"download"}',6,6
Change-Id: I3b4399ae924dadb89875735b12a2f92453b6754c
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-create.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-query.test
8 files changed, 112 insertions(+), 76 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/18641/1
--
To view, visit http://gerrit.cloudera.org:8080/18641
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I3b4399ae924dadb89875735b12a2f92453b6754c
Gerrit-Change-Number: 18641
Gerrit-PatchSet: 1
Gerrit-Owner: Xiaoqing Gao
[Impala-ASF-CR] IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18641
to look at the new patch set (#2).
Change subject: IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the
partitions
..
IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions
Currently, SHOW PARTITIONS on Iceberg tables only outputs the partition
spec which is not too useful.
Instead it should output the concrete partitions, number of files, number
of rows in each partitions. E.g.:
SHOW PARTITIONS iceberg_partitioned;
'{"event_time_hour":"438296","action":"view"}',8,8
'{"event_time_hour":"438297","action":"click"}',6,6
'{"event_time_hour":"438298","action":"download"}',6,6
Change-Id: I3b4399ae924dadb89875735b12a2f92453b6754c
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-create.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-query.test
8 files changed, 111 insertions(+), 76 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/18641/2
--
To view, visit http://gerrit.cloudera.org:8080/18641
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3b4399ae924dadb89875735b12a2f92453b6754c
Gerrit-Change-Number: 18641
Gerrit-PatchSet: 2
Gerrit-Owner: Xiaoqing Gao
Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18641 ) Change subject: IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10817/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18641 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3b4399ae924dadb89875735b12a2f92453b6754c Gerrit-Change-Number: 18641 Gerrit-PatchSet: 1 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 21 Jun 2022 03:48:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18641 ) Change subject: IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10818/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18641 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3b4399ae924dadb89875735b12a2f92453b6754c Gerrit-Change-Number: 18641 Gerrit-PatchSet: 2 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 21 Jun 2022 03:57:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18641
to look at the new patch set (#3).
Change subject: IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the
partitions
..
IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions
Currently, SHOW PARTITIONS on Iceberg tables only outputs the partition
spec which is not too useful.
Instead it should output the concrete partitions, number of files, number
of rows in each partitions. E.g.:
SHOW PARTITIONS iceberg_partitioned;
'{"event_time_hour":"438296","action":"view"}',8,8
'{"event_time_hour":"438297","action":"click"}',6,6
'{"event_time_hour":"438298","action":"download"}',6,6
Change-Id: I3b4399ae924dadb89875735b12a2f92453b6754c
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-create.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-query.test
8 files changed, 111 insertions(+), 76 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/18641/3
--
To view, visit http://gerrit.cloudera.org:8080/18641
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3b4399ae924dadb89875735b12a2f92453b6754c
Gerrit-Change-Number: 18641
Gerrit-PatchSet: 3
Gerrit-Owner: Xiaoqing Gao
Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18641
to look at the new patch set (#4).
Change subject: IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the
partitions
..
IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions
Currently, SHOW PARTITIONS on Iceberg tables only outputs the partition
spec which is not too useful.
Instead it should output the concrete partitions, number of files, number
of rows in each partitions. E.g.:
SHOW PARTITIONS iceberg_partitioned;
'{"event_time_hour":"438296","action":"view"}',8,8
'{"event_time_hour":"438297","action":"click"}',6,6
'{"event_time_hour":"438298","action":"download"}',6,6
Change-Id: I3b4399ae924dadb89875735b12a2f92453b6754c
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-create.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-query.test
8 files changed, 107 insertions(+), 76 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/18641/4
--
To view, visit http://gerrit.cloudera.org:8080/18641
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3b4399ae924dadb89875735b12a2f92453b6754c
Gerrit-Change-Number: 18641
Gerrit-PatchSet: 4
Gerrit-Owner: Xiaoqing Gao
Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18641 ) Change subject: IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions .. Patch Set 3: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/10819/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/18641 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3b4399ae924dadb89875735b12a2f92453b6754c Gerrit-Change-Number: 18641 Gerrit-PatchSet: 3 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 21 Jun 2022 04:25:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18641 ) Change subject: IMPALA-11320: SHOW PARTITIONS on Iceberg table doesn't list the partitions .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10820/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18641 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3b4399ae924dadb89875735b12a2f92453b6754c Gerrit-Change-Number: 18641 Gerrit-PatchSet: 4 Gerrit-Owner: Xiaoqing Gao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 21 Jun 2022 04:40:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11341: Print error log files when data-loading fails
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18597 ) Change subject: IMPALA-11341: Print error log files when data-loading fails .. IMPALA-11341: Print error log files when data-loading fails run-step will print the last 50 lines of the log file when the step fails. However, the data-loading log file just shows the log file names of the failed sql files. We still need to manually look into them for more details. This patch extends run-step to also print the last 100 lines of such log files. Tested locally by stopping impala cluster to mimic an error state and run these in my terminal: source testdata/bin/run-step.sh export LOG_DIR=tmp run-step "Loading functional-query data" load-functional-query.log bin/load-data.py -e exhaustive -w functional-query Change-Id: I771b4b74e00d89a3cc85f26b8b1da66c3dd66304 Reviewed-on: http://gerrit.cloudera.org:8080/18597 Reviewed-by: Quanlong Huang Tested-by: Impala Public Jenkins --- M testdata/bin/run-step.sh 1 file changed, 11 insertions(+), 1 deletion(-) Approvals: Quanlong Huang: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/18597 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I771b4b74e00d89a3cc85f26b8b1da66c3dd66304 Gerrit-Change-Number: 18597 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto
[Impala-ASF-CR] IMPALA-11341: Print error log files when data-loading fails
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18597 ) Change subject: IMPALA-11341: Print error log files when data-loading fails .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/18597 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I771b4b74e00d89a3cc85f26b8b1da66c3dd66304 Gerrit-Change-Number: 18597 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Tue, 21 Jun 2022 06:51:36 + Gerrit-HasComments: No
