[Impala-ASF-CR] IMPALA-12433: Share buffers among channels in KrpcDataStreamSender
Zihao Ye has posted comments on this change. ( http://gerrit.cloudera.org:8080/20719 ) Change subject: IMPALA-12433: Share buffers among channels in KrpcDataStreamSender .. Patch Set 12: Code-Review+1 (2 comments) Looks great! http://gerrit.cloudera.org:8080/#/c/20719/12/be/src/runtime/krpc-data-stream-sender.cc File be/src/runtime/krpc-data-stream-sender.cc: http://gerrit.cloudera.org:8080/#/c/20719/12/be/src/runtime/krpc-data-stream-sender.cc@1056 PS12, Line 1056: nit: redundant space http://gerrit.cloudera.org:8080/#/c/20719/12/be/src/runtime/row-batch.cc File be/src/runtime/row-batch.cc: http://gerrit.cloudera.org:8080/#/c/20719/12/be/src/runtime/row-batch.cc@280 PS12, Line 280: _ nit: redundant underline -- To view, visit http://gerrit.cloudera.org:8080/20719 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I64854a350a9dae8bf3af11c871882ea4750e60b3 Gerrit-Change-Number: 20719 Gerrit-PatchSet: 12 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Fri, 02 Feb 2024 07:02:06 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12718: Provides UTF-8 support for the trim functions
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20926 ) Change subject: IMPALA-12718: Provides UTF-8 support for the trim functions .. Patch Set 12: Code-Review+2 Carry Riza's +1. -- To view, visit http://gerrit.cloudera.org:8080/20926 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cfaffd71009f16eae75910af835bd2a34410856 Gerrit-Change-Number: 20926 Gerrit-PatchSet: 12 Gerrit-Owner: Zihao Ye Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Fri, 02 Feb 2024 06:35:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12718: Provides UTF-8 support for the trim functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20926 ) Change subject: IMPALA-12718: Provides UTF-8 support for the trim functions .. Patch Set 13: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10228/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/20926 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cfaffd71009f16eae75910af835bd2a34410856 Gerrit-Change-Number: 20926 Gerrit-PatchSet: 13 Gerrit-Owner: Zihao Ye Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Fri, 02 Feb 2024 06:36:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12718: Provides UTF-8 support for the trim functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20926 ) Change subject: IMPALA-12718: Provides UTF-8 support for the trim functions .. Patch Set 13: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/20926 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cfaffd71009f16eae75910af835bd2a34410856 Gerrit-Change-Number: 20926 Gerrit-PatchSet: 13 Gerrit-Owner: Zihao Ye Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Fri, 02 Feb 2024 06:36:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12718: Provides UTF-8 support for the trim functions
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/20926 ) Change subject: IMPALA-12718: Provides UTF-8 support for the trim functions .. Patch Set 12: Code-Review+1 (1 comment) Thank you! Everything else looks good to me. http://gerrit.cloudera.org:8080/#/c/20926/12/be/src/exprs/expr-test.cc File be/src/exprs/expr-test.cc: http://gerrit.cloudera.org:8080/#/c/20926/12/be/src/exprs/expr-test.cc@1 PS12, Line 1: // Tests *trim() with UTF-8 characters in UTF8_MODE. Done -- To view, visit http://gerrit.cloudera.org:8080/20926 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cfaffd71009f16eae75910af835bd2a34410856 Gerrit-Change-Number: 20926 Gerrit-PatchSet: 12 Gerrit-Owner: Zihao Ye Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Fri, 02 Feb 2024 05:50:13 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12780: Only show non-default options in the catalog operations page
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20985 ) Change subject: IMPALA-12780: Only show non-default options in the catalog operations page .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15143/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20985 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie62d4b9e4d357e02764e7a62f4dc107de602e1a5 Gerrit-Change-Number: 20985 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 02 Feb 2024 05:07:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12780: Only show non-default options in the catalog operations page
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20985 Change subject: IMPALA-12780: Only show non-default options in the catalog operations page .. IMPALA-12780: Only show non-default options in the catalog operations page The details shown in the catalog operations page are too verbose. For instance, for ExecDdlRequest, we show the string of TDdlQueryOptions: query_options=TDdlQueryOptions(sync_ddl:false, debug_action:, lock_max_wait_time_s:300, kudu_table_reserve_seconds:0) What really matters is the non-default options, e.g. if sync_ddl is set to true, it should be shown. This patch improve the details field to show only non-default options. Also wraps the content of Query Id and Details columns so they can fit into the table. Change-Id: Ie62d4b9e4d357e02764e7a62f4dc107de602e1a5 --- M fe/src/main/java/org/apache/impala/catalog/monitor/CatalogOperationTracker.java M www/catalog_operations.tmpl 2 files changed, 49 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/20985/1 -- To view, visit http://gerrit.cloudera.org:8080/20985 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ie62d4b9e4d357e02764e7a62f4dc107de602e1a5 Gerrit-Change-Number: 20985 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang
[Impala-ASF-CR] IMPALA-12718: Provides UTF-8 support for the trim functions
Zihao Ye has posted comments on this change. ( http://gerrit.cloudera.org:8080/20926 ) Change subject: IMPALA-12718: Provides UTF-8 support for the trim functions .. Patch Set 11: (1 comment) Thank you for the code review! http://gerrit.cloudera.org:8080/#/c/20926/11/be/src/exprs/expr-test.cc File be/src/exprs/expr-test.cc: http://gerrit.cloudera.org:8080/#/c/20926/11/be/src/exprs/expr-test.cc@1 PS11, Line 1: // Tests *trim() with UTF-8 characters in UTF8_MODE. > Please add trim, ltrim, rtrim, and btrim tests without second argument unde Done -- To view, visit http://gerrit.cloudera.org:8080/20926 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cfaffd71009f16eae75910af835bd2a34410856 Gerrit-Change-Number: 20926 Gerrit-PatchSet: 11 Gerrit-Owner: Zihao Ye Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Fri, 02 Feb 2024 04:31:48 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12778: Fix time strings in catalog operations page
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20983 ) Change subject: IMPALA-12778: Fix time strings in catalog operations page .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10227/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/20983 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibebfc85267aabe9cef3a53f487d0ba53e050aa4c Gerrit-Change-Number: 20983 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Fri, 02 Feb 2024 04:22:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12718: Provides UTF-8 support for the trim functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20926 ) Change subject: IMPALA-12718: Provides UTF-8 support for the trim functions .. Patch Set 12: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15142/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20926 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cfaffd71009f16eae75910af835bd2a34410856 Gerrit-Change-Number: 20926 Gerrit-PatchSet: 12 Gerrit-Owner: Zihao Ye Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Fri, 02 Feb 2024 04:06:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12770: Fix infinite loop for nested Case expressions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20982 ) Change subject: IMPALA-12770: Fix infinite loop for nested Case expressions .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15141/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20982 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaea1b3a35967f92b3e4f3445a378ed58adaa7da5 Gerrit-Change-Number: 20982 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 02 Feb 2024 04:02:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12778: Fix time strings in catalog operations page
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/20983 ) Change subject: IMPALA-12778: Fix time strings in catalog operations page .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/20983 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibebfc85267aabe9cef3a53f487d0ba53e050aa4c Gerrit-Change-Number: 20983 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Fri, 02 Feb 2024 03:45:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12718: Provides UTF-8 support for the trim functions
Hello Quanlong Huang, Riza Suminto, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20926 to look at the new patch set (#12). Change subject: IMPALA-12718: Provides UTF-8 support for the trim functions .. IMPALA-12718: Provides UTF-8 support for the trim functions Currently, the trim function (including BTRIM, LTRIM, RTRIM) cannot correctly handle strings containing multi-byte UTF-8 characters. Multi-byte UTF-8 characters are interpreted as multiple single-byte characters, leading to unexpected results. This patch provides UTF-8 support for the trim functions, enabling these functions to correctly handle multi-byte UTF-8 characters (when set utf8_mode=true). It also introduces a set of trim functions with the 'utf8_' prefix, offering the same capability even when utf8_mode is not enabled. Testing: - Added new BE test case in ExprTest#Utf8Test - Added new E2E test case in TestUtf8StringFunctions Change-Id: I5cfaffd71009f16eae75910af835bd2a34410856 --- M be/src/exprs/expr-test.cc M be/src/exprs/string-functions-ir.cc M be/src/exprs/string-functions.h M be/src/util/bit-util.h M be/src/util/string-util.cc M common/function-registry/impala_functions.py M testdata/workloads/functional-query/queries/QueryTest/utf8-string-functions.test 7 files changed, 342 insertions(+), 39 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/20926/12 -- To view, visit http://gerrit.cloudera.org:8080/20926 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5cfaffd71009f16eae75910af835bd2a34410856 Gerrit-Change-Number: 20926 Gerrit-PatchSet: 12 Gerrit-Owner: Zihao Ye Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Zihao Ye
[Impala-ASF-CR] IMPALA-12778: Fix time strings in catalog operations page
Zihao Ye has posted comments on this change. ( http://gerrit.cloudera.org:8080/20983 ) Change subject: IMPALA-12778: Fix time strings in catalog operations page .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/20983 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibebfc85267aabe9cef3a53f487d0ba53e050aa4c Gerrit-Change-Number: 20983 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Fri, 02 Feb 2024 03:37:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12770: Fix infinite loop for nested Case expressions
Wenzhe Zhou has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/20982 ) Change subject: IMPALA-12770: Fix infinite loop for nested Case expressions .. IMPALA-12770: Fix infinite loop for nested Case expressions ExprRewriter enter infinite loop when run following query with nested Case expressions: select case case '' when 'abc' then t4.string_col end when 'none' then 'Total' end as fcol from functional.alltypes as t4 limit 1; The 'case' expr is set as literal NULL expression for outer case expression since all 'when' expr are false in inner case expression. This causes function SimplifyConditionalsRule.simplifyCaseExpr() to be called recusively and enter infinite loop. This patch fixs the issue by not rewriting a case expression if 'case' expr is literal NULL expression. Testing: - Manually ran above query successfully. - Added new test cases in ExprRewriteRulesTest.testCaseWithExpr. - Passed core-tests. Change-Id: Iaea1b3a35967f92b3e4f3445a378ed58adaa7da5 --- M fe/src/main/java/org/apache/impala/rewrite/SimplifyConditionalsRule.java M fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java 2 files changed, 16 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/20982/2 -- To view, visit http://gerrit.cloudera.org:8080/20982 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iaea1b3a35967f92b3e4f3445a378ed58adaa7da5 Gerrit-Change-Number: 20982 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-12778: Fix time strings in catalog operations page
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20983 ) Change subject: IMPALA-12778: Fix time strings in catalog operations page .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15140/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20983 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibebfc85267aabe9cef3a53f487d0ba53e050aa4c Gerrit-Change-Number: 20983 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 02 Feb 2024 03:10:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12770: Fix infinite loop for nested Case expressions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20982 ) Change subject: IMPALA-12770: Fix infinite loop for nested Case expressions .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15139/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20982 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaea1b3a35967f92b3e4f3445a378ed58adaa7da5 Gerrit-Change-Number: 20982 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 02 Feb 2024 03:00:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12778: Fix time strings in catalog operations page
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20983 Change subject: IMPALA-12778: Fix time strings in catalog operations page .. IMPALA-12778: Fix time strings in catalog operations page In the /queries page of impalad, we are showing time strings using the server timezone. However, in the /operations page of catalogd, UTC timezone is used, which is confusing. This fixes the /operations page to use the server timezone as well. Tests - Add e2e test Change-Id: Ibebfc85267aabe9cef3a53f487d0ba53e050aa4c --- M be/src/catalog/catalog-server.cc M tests/webserver/test_web_pages.py 2 files changed, 41 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/20983/1 -- To view, visit http://gerrit.cloudera.org:8080/20983 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ibebfc85267aabe9cef3a53f487d0ba53e050aa4c Gerrit-Change-Number: 20983 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang
[Impala-ASF-CR] IMPALA-12770: Fix infinite loop for nested Case expressions
Wenzhe Zhou has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20982 Change subject: IMPALA-12770: Fix infinite loop for nested Case expressions .. IMPALA-12770: Fix infinite loop for nested Case expressions ExprRewriter enter infinite loop when run following query with nested Case expressions: select case case '' when 'abc' then t4.string_col end when 'none' then 'Total' end as fcol from functional.alltypes as t4 limit 1; The 'case' expr is set as literal NULL expression for outer case expression since all 'when' expr are false in inner case expression. This causes function SimplifyConditionalsRule.simplifyCaseExpr() to be called recusively and exter infinite loop. This patch fixs the issue by not rewriting a case expression if 'case' expr is literal NULL expression. Testing: - Manually ran above query successfully. - Added new test cases in ExprRewriteRulesTest.testCaseWithExpr. - Passed core-tests. Change-Id: Iaea1b3a35967f92b3e4f3445a378ed58adaa7da5 --- M fe/src/main/java/org/apache/impala/rewrite/SimplifyConditionalsRule.java M fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java 2 files changed, 16 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/82/20982/1 -- To view, visit http://gerrit.cloudera.org:8080/20982 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Iaea1b3a35967f92b3e4f3445a378ed58adaa7da5 Gerrit-Change-Number: 20982 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-12607: Bump the GBN and fetch events specific to the db/table from the metastore
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20979 ) Change subject: IMPALA-12607: Bump the GBN and fetch events specific to the db/table from the metastore .. Patch Set 1: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/15138/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/20979 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6aecd5108b31c24e6e2c6f9fba6d4d44a3b00729 Gerrit-Change-Number: 20979 Gerrit-PatchSet: 1 Gerrit-Owner: Sai Hemanth Gantasala Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Feb 2024 23:35:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12607: Bump the GBN and fetch events specific to the db/table from the metastore
Sai Hemanth Gantasala has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20979 Change subject: IMPALA-12607: Bump the GBN and fetch events specific to the db/table from the metastore .. IMPALA-12607: Bump the GBN and fetch events specific to the db/table from the metastore Bump the GBN to 49623641 to leverage HIVE-27499, so that Impala can directly fetch the latest events specific to the db/table from the metastore, instead of fetching the events from metastore and then filtering in the cache matching the DbName/TableName. Implementation Details: Currently when a DDL/DML is performed in Impala, we fetch all the events from metastore based on current eventId and then filter them in Impala which can be a bottleneck if the events count is huge. This can be optimized by including db name and/or table name in the notification event request object and then filter by event type in impala. This can provide performance boost on tables that generate a lot of events. Testing: 1) Did some tests in local cluster 2) Added a test case in MetaStoreEventsProcessorTest Change-Id: I6aecd5108b31c24e6e2c6f9fba6d4d44a3b00729 --- M bin/impala-config.sh M fe/src/main/java/org/apache/impala/catalog/TableLoader.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServiceHandler.java M fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 7 files changed, 207 insertions(+), 41 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/79/20979/1 -- To view, visit http://gerrit.cloudera.org:8080/20979 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I6aecd5108b31c24e6e2c6f9fba6d4d44a3b00729 Gerrit-Change-Number: 20979 Gerrit-PatchSet: 1 Gerrit-Owner: Sai Hemanth Gantasala
[Impala-ASF-CR] IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20204 ) Change subject: IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile .. Patch Set 11: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/10226/ -- To view, visit http://gerrit.cloudera.org:8080/20204 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee0b4967429ea733729ad8e44df32e3b24b88525 Gerrit-Change-Number: 20204 Gerrit-PatchSet: 11 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 01 Feb 2024 22:17:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12718: Provides UTF-8 support for the trim functions
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/20926 ) Change subject: IMPALA-12718: Provides UTF-8 support for the trim functions .. Patch Set 11: (1 comment) http://gerrit.cloudera.org:8080/#/c/20926/11/be/src/exprs/expr-test.cc File be/src/exprs/expr-test.cc: http://gerrit.cloudera.org:8080/#/c/20926/11/be/src/exprs/expr-test.cc@1 PS11, Line 1: // Tests *trim() with UTF-8 characters in UTF8_MODE. Please add trim, ltrim, rtrim, and btrim tests without second argument under utf8_mode. -- To view, visit http://gerrit.cloudera.org:8080/20926 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cfaffd71009f16eae75910af835bd2a34410856 Gerrit-Change-Number: 20926 Gerrit-PatchSet: 11 Gerrit-Owner: Zihao Ye Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Zihao Ye Gerrit-Comment-Date: Thu, 01 Feb 2024 20:27:29 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12503: Support date data type for predicates for external data source table
Wenzhe Zhou has posted comments on this change. (
http://gerrit.cloudera.org:8080/20915 )
Change subject: IMPALA-12503: Support date data type for predicates for
external data source table
..
Patch Set 13:
(3 comments)
Please add a test case for '=' with value like '1990-01-01', expect return 0
row. Add similar test cases for '<' and '>' to return 0 row.
http://gerrit.cloudera.org:8080/#/c/20915/13/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/DatabaseAccessor.java
File
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/DatabaseAccessor.java:
http://gerrit.cloudera.org:8080/#/c/20915/13/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/DatabaseAccessor.java@36
PS13, Line 36: dateCol
nit: rename to dateVal
http://gerrit.cloudera.org:8080/#/c/20915/13/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java
File
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java:
http://gerrit.cloudera.org:8080/#/c/20915/13/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java@218
PS13, Line 218: dateCol
nit: rename to dateVal
http://gerrit.cloudera.org:8080/#/c/20915/13/testdata/workloads/functional-query/queries/QueryTest/impala-ext-jdbc-tables-predicates.test
File
testdata/workloads/functional-query/queries/QueryTest/impala-ext-jdbc-tables-predicates.test:
http://gerrit.cloudera.org:8080/#/c/20915/13/testdata/workloads/functional-query/queries/QueryTest/impala-ext-jdbc-tables-predicates.test@49
PS13, Line 49: QUERY
: # Create external JDBC DataSource table
: DROP TABLE IF EXISTS alltypes_jdbc_datasource_2;
: CREATE TABLE alltypes_jdbc_datasource_2 (
: id INT,
: bool_col BOOLEAN,
: tinyint_col TINYINT,
: smallint_col SMALLINT,
: int_col INT,
: bigint_col BIGINT,
: float_col FLOAT,
: double_col DOUBLE,
: date_col DATE,
: string_col STRING,
: timestamp_col TIMESTAMP)
: PRODUCED BY DATA SOURCE TestJdbcDataSource(
: '{"database.type":"IMPALA",
:
"jdbc.url":"jdbc:impala://$INTERNAL_LISTEN_HOST:21050/functional",
: "jdbc.auth":"AuthMech=0",
: "jdbc.driver":"com.cloudera.impala.jdbc.Driver",
:
"driver.url":"$FILESYSTEM_PREFIX/test-warehouse/data-sources/jdbc-drivers/ImpalaJDBC42.jar",
: "dbcp.username":"impala",
: "dbcp.password":"cloudera",
: "table":"alltypes_with_date"}');
: RESULTS
: 'Table has been created.'
still not removed
--
To view, visit http://gerrit.cloudera.org:8080/20915
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibf13cbefaad812a0f78755c5791d82b24a3395e4
Gerrit-Change-Number: 20915
Gerrit-PatchSet: 13
Gerrit-Owner: gaurav singh
Gerrit-Reviewer: Abhishek Rawat
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Wenzhe Zhou
Gerrit-Reviewer: gaurav singh
Gerrit-Comment-Date: Thu, 01 Feb 2024 19:45:13 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12503: Support date data type for predicates for external data source table
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20915 ) Change subject: IMPALA-12503: Support date data type for predicates for external data source table .. Patch Set 13: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15137/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20915 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibf13cbefaad812a0f78755c5791d82b24a3395e4 Gerrit-Change-Number: 20915 Gerrit-PatchSet: 13 Gerrit-Owner: gaurav singh Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: gaurav singh Gerrit-Comment-Date: Thu, 01 Feb 2024 19:37:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12503: Support date data type for predicates for external data source table
gaurav singh has posted comments on this change. (
http://gerrit.cloudera.org:8080/20915 )
Change subject: IMPALA-12503: Support date data type for predicates for
external data source table
..
Patch Set 13:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/20915/12/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/DatabaseAccessor.java
File
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/DatabaseAccessor.java:
http://gerrit.cloudera.org:8080/#/c/20915/12/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/DatabaseAccessor.java@36
PS12, Line 36: getDateString(int dateCol);
> rename function and input parameter as getDateString(int dateVal)
Done
http://gerrit.cloudera.org:8080/#/c/20915/12/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java
File
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java:
http://gerrit.cloudera.org:8080/#/c/20915/12/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java@218
PS12, Line 218: getDateString(int dateCol)
> rename function and input parameter as getDateString(int dateVal)
Done
http://gerrit.cloudera.org:8080/#/c/20915/12/testdata/workloads/functional-query/queries/QueryTest/impala-ext-jdbc-tables-predicates.test
File
testdata/workloads/functional-query/queries/QueryTest/impala-ext-jdbc-tables-predicates.test:
http://gerrit.cloudera.org:8080/#/c/20915/12/testdata/workloads/functional-query/queries/QueryTest/impala-ext-jdbc-tables-predicates.test@49
PS12, Line 49: QUERY
: # Create external JDBC DataSource table
: DROP TABLE IF EXISTS alltypes_jdbc_datasource_2;
: CREATE TABLE alltypes_jdbc_datasource_2 (
: id INT,
: bool_col BOOLEAN,
: tinyint_col TINYINT,
: smallint_col SMALLINT,
: int_col INT,
: bigint_col BIGINT,
: float_col FLOAT,
: double_col DOUBLE,
: date_col DATE,
: string_col STRING,
: timestamp_col TIMESTAMP)
: PRODUCED BY DATA SOURCE TestJdbcDataSource(
: '{"database.type":"IMPALA",
:
"jdbc.url":"jdbc:impala://$INTERNAL_LISTEN_HOST:21050/functional",
: "jdbc.auth":"AuthMech=0",
: "jdbc.driver":"com.cloudera.impala.jdbc.Driver",
:
"driver.url":"$FILESYSTEM_PREFIX/test-warehouse/data-sources/jdbc-drivers/ImpalaJDBC42.jar",
: "dbcp.username":"impala",
: "dbcp.password":"cloudera",
: "table":"alltypes_with_date"}');
: RESULTS
: 'Table has been created.'
:
> remove this table since it is not used
Done
http://gerrit.cloudera.org:8080/#/c/20915/12/testdata/workloads/functional-query/queries/QueryTest/mysql-ext-jdbc-tables.test
File
testdata/workloads/functional-query/queries/QueryTest/mysql-ext-jdbc-tables.test:
http://gerrit.cloudera.org:8080/#/c/20915/12/testdata/workloads/functional-query/queries/QueryTest/mysql-ext-jdbc-tables.test@185
PS12, Line 185: 2009-01-09
> nit: just curious why the values of date columns in MySQL database are diff
My guess would be that the engines have different data structures for data
representation ?
--
To view, visit http://gerrit.cloudera.org:8080/20915
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibf13cbefaad812a0f78755c5791d82b24a3395e4
Gerrit-Change-Number: 20915
Gerrit-PatchSet: 13
Gerrit-Owner: gaurav singh
Gerrit-Reviewer: Abhishek Rawat
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Wenzhe Zhou
Gerrit-Reviewer: gaurav singh
Gerrit-Comment-Date: Thu, 01 Feb 2024 19:10:45 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12503: Support date data type for predicates for external data source table
Hello Abhishek Rawat, Wenzhe Zhou, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20915 to look at the new patch set (#13). Change subject: IMPALA-12503: Support date data type for predicates for external data source table .. IMPALA-12503: Support date data type for predicates for external data source table This patch adds support for datatype date as predicates for external data sources. Testing: - Added tests for date predicates with operators: '=', '>', '<', '>=', '<=', '!=', 'BETWEEN'. Change-Id: Ibf13cbefaad812a0f78755c5791d82b24a3395e4 --- M fe/src/main/java/org/apache/impala/planner/DataSourceScanNode.java M java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/JdbcDataSource.java M java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/DatabaseAccessor.java M java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java M java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/util/QueryConditionUtil.java M testdata/bin/create-ext-data-source-table.sql M testdata/bin/load-ext-data-sources.sh M testdata/bin/setup-mysql-env.sh A testdata/workloads/functional-query/queries/QueryTest/impala-ext-jdbc-tables-predicates.test M testdata/workloads/functional-query/queries/QueryTest/jdbc-data-source.test M testdata/workloads/functional-query/queries/QueryTest/mysql-ext-jdbc-tables.test M tests/custom_cluster/test_ext_data_sources.py 12 files changed, 505 insertions(+), 57 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/15/20915/13 -- To view, visit http://gerrit.cloudera.org:8080/20915 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibf13cbefaad812a0f78755c5791d82b24a3395e4 Gerrit-Change-Number: 20915 Gerrit-PatchSet: 13 Gerrit-Owner: gaurav singh Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: gaurav singh
[Impala-ASF-CR] IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/20204 ) Change subject: IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile .. Patch Set 11: Code-Review+1 (2 comments) Thanks for the clarifications Zoltan. LGTM! http://gerrit.cloudera.org:8080/#/c/20204/9/fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java File fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java: http://gerrit.cloudera.org:8080/#/c/20204/9/fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java@132 PS9, Line 132: 0 nit: in Impala I have found -1s as invalid snapshot ids. http://gerrit.cloudera.org:8080/#/c/20204/9/fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java@145 PS9, Line 145: IcebergUtil.getSnapshotId(getIceTable(), tblRef_.getTimeTravelSpec()); > An IcebergScanPlanner can create up to 3 IcebergScanNode objects, so we can Thanks for the explanation Zoltan, it totally makes sense to keep it as is. My intent with the comment was to keep the constructor light if possible. In case someone in the future has to instantiate an IcebergScanNode in a place where it is difficult to obtain. -- To view, visit http://gerrit.cloudera.org:8080/20204 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee0b4967429ea733729ad8e44df32e3b24b88525 Gerrit-Change-Number: 20204 Gerrit-PatchSet: 11 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 01 Feb 2024 18:41:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/20922 ) Change subject: IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest .. Patch Set 6: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/20922 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaffddd70c2da8376ca6c40f65606bbac46c34de7 Gerrit-Change-Number: 20922 Gerrit-PatchSet: 6 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 01 Feb 2024 18:34:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/20922 ) Change subject: IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/20922/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/20922/5//COMMIT_MSG@32 PS5, Line 32: ar > nit: are Done -- To view, visit http://gerrit.cloudera.org:8080/20922 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaffddd70c2da8376ca6c40f65606bbac46c34de7 Gerrit-Change-Number: 20922 Gerrit-PatchSet: 6 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 01 Feb 2024 18:33:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest
Hello Quanlong Huang, Abhishek Rawat, Wenzhe Zhou, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20922 to look at the new patch set (#6). Change subject: IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest .. IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest Querying against large-scale databases is a good way for testing Impala. However, it is impractical to do in a single-node development machine. Frontend testing does not run the test query in the backend executor and can benefit from simulated large-scale test cases. This patch attempts to do it by instrumenting the CatalogD metadata loading code to scale tpcds_partitioned_parquet_snap by injecting column stats from a 3TB TPC-DS dataset in TpcdsCpuCostPlannerTest. The large-scale column stats are expressed in stats-3TB.json, taken by running "SHOW COLUMN STATS" and "DESCRIBE FORMATTED" queries on a 3TB dataset loaded using impala-tpcds-kit. It is parsed and then piggy-backed through RuntimeEnv. Code that populates stats metadata (caller of FeCatalogUtils.getRowCount(), FeCatalogUtils.getTotalSize(), and FeCatalogUtils.injectColumnStats()) are instrumented to populate stats from RuntimeEnv instead of Metastore. Scaled-up tables are invalidated before a test run to reload them with new high-scale stats. This patch also adds a scan range limit injection to force ScanNode over a single file table to act as if it scans a multi-files table. tpcds_partitioned_schema_template.sql is modified to match column names and types from impala-tpcds-kit. The test files under PlannerTest/tpcds_cpu_cost/ are replaced with queries that are specifically generated to run against the 3TB scale factor of the TPC-DS dataset (https://github.com/cloudera/impala-tpcds-kit/blob/separate_queries_per_scale_factor/queries/sf3000/). All query plans match with query plans obtained through real query runs in a large cluster except for a few mismatches due to the hard limit on the number of files at a table. Below are 3 queries out of 103 that still do not have a matching shape and the reasons. +-+--+ | Q | Reason | +-+--+ | 10a | different num files in customer_demographics | | 34 | different num files in customer | | 69 | different num files in customer | +-+--+ Testing: - Scale tables of tpcds_partitioned_parquet_snap in TpcdsCpuCostPlannerTest to simulate 3TB TPC-DS. The number of executors is raised from 3 to 10, and REPLICA_PREFERENCE=REMOTE to ignore data locality. - Pass core tests. Change-Id: Iaffddd70c2da8376ca6c40f65606bbac46c34de7 --- M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java A fe/src/main/java/org/apache/impala/catalog/SideloadTableStats.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java M fe/src/main/java/org/apache/impala/catalog/local/LocalTable.java M fe/src/main/java/org/apache/impala/common/RuntimeEnv.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java M fe/src/test/java/org/apache/impala/planner/TpcdsCpuCostPlannerTest.java A fe/src/test/java/org/apache/impala/testutil/StatsJsonParser.java M testdata/datasets/tpcds_partitioned/tpcds_partitioned_schema_template.sql A testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/stats-3TB.json M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q01.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q02.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q03.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q04.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q05.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q06.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q07.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q08.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q09.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q10a.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q11.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q12.test M testdata/workloads/functional-planner/querie
[Impala-ASF-CR] IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/20922 ) Change subject: IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest .. Patch Set 5: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/20922/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/20922/5//COMMIT_MSG@32 PS5, Line 32: is nit: are -- To view, visit http://gerrit.cloudera.org:8080/20922 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaffddd70c2da8376ca6c40f65606bbac46c34de7 Gerrit-Change-Number: 20922 Gerrit-PatchSet: 5 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 01 Feb 2024 18:23:31 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20204 ) Change subject: IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile .. Patch Set 11: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15136/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20204 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee0b4967429ea733729ad8e44df32e3b24b88525 Gerrit-Change-Number: 20204 Gerrit-PatchSet: 11 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 01 Feb 2024 18:10:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/20922 ) Change subject: IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/20922/4/fe/src/main/java/org/apache/impala/common/RuntimeEnv.java File fe/src/main/java/org/apache/impala/common/RuntimeEnv.java: http://gerrit.cloudera.org:8080/#/c/20922/4/fe/src/main/java/org/apache/impala/common/RuntimeEnv.java@39 PS4, Line 39: : // Map of > that is used to simula > Looking around org/apache/impala/common, this package is pretty liberal in Done -- To view, visit http://gerrit.cloudera.org:8080/20922 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaffddd70c2da8376ca6c40f65606bbac46c34de7 Gerrit-Change-Number: 20922 Gerrit-PatchSet: 5 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 01 Feb 2024 18:05:29 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20922 ) Change subject: IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15135/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20922 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaffddd70c2da8376ca6c40f65606bbac46c34de7 Gerrit-Change-Number: 20922 Gerrit-PatchSet: 5 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 01 Feb 2024 18:05:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12503: Support date data type for predicates for external data source table
Wenzhe Zhou has posted comments on this change. (
http://gerrit.cloudera.org:8080/20915 )
Change subject: IMPALA-12503: Support date data type for predicates for
external data source table
..
Patch Set 12:
(6 comments)
Overall looks goods, just some minor issues.
http://gerrit.cloudera.org:8080/#/c/20915/12//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20915/12//COMMIT_MSG@14
PS12, Line 14: binary
nit: remove binary
http://gerrit.cloudera.org:8080/#/c/20915/12//COMMIT_MSG@15
PS12, Line 15: =
test cases for '=' are missing
http://gerrit.cloudera.org:8080/#/c/20915/12/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/DatabaseAccessor.java
File
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/DatabaseAccessor.java:
http://gerrit.cloudera.org:8080/#/c/20915/12/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/DatabaseAccessor.java@36
PS12, Line 36: getDateColumn(int dateCol);
rename function and input parameter as getDateString(int dateVal)
http://gerrit.cloudera.org:8080/#/c/20915/12/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java
File
java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java:
http://gerrit.cloudera.org:8080/#/c/20915/12/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java@218
PS12, Line 218: getDateColumn(int dateCol)
rename function and input parameter as getDateString(int dateVal)
http://gerrit.cloudera.org:8080/#/c/20915/12/testdata/workloads/functional-query/queries/QueryTest/impala-ext-jdbc-tables-predicates.test
File
testdata/workloads/functional-query/queries/QueryTest/impala-ext-jdbc-tables-predicates.test:
http://gerrit.cloudera.org:8080/#/c/20915/12/testdata/workloads/functional-query/queries/QueryTest/impala-ext-jdbc-tables-predicates.test@49
PS12, Line 49: QUERY
: # Create external JDBC DataSource table
: DROP TABLE IF EXISTS alltypes_jdbc_datasource_2;
: CREATE TABLE alltypes_jdbc_datasource_2 (
: id INT,
: bool_col BOOLEAN,
: tinyint_col TINYINT,
: smallint_col SMALLINT,
: int_col INT,
: bigint_col BIGINT,
: float_col FLOAT,
: double_col DOUBLE,
: date_col DATE,
: string_col STRING,
: timestamp_col TIMESTAMP)
: PRODUCED BY DATA SOURCE TestJdbcDataSource(
: '{"database.type":"IMPALA",
:
"jdbc.url":"jdbc:impala://$INTERNAL_LISTEN_HOST:21050/functional",
: "jdbc.auth":"AuthMech=0",
: "jdbc.driver":"com.cloudera.impala.jdbc.Driver",
:
"driver.url":"$FILESYSTEM_PREFIX/test-warehouse/data-sources/jdbc-drivers/ImpalaJDBC42.jar",
: "dbcp.username":"impala",
: "dbcp.password":"cloudera",
: "table":"alltypes_with_date"}');
: RESULTS
: 'Table has been created.'
:
remove this table since it is not used
http://gerrit.cloudera.org:8080/#/c/20915/12/testdata/workloads/functional-query/queries/QueryTest/mysql-ext-jdbc-tables.test
File
testdata/workloads/functional-query/queries/QueryTest/mysql-ext-jdbc-tables.test:
http://gerrit.cloudera.org:8080/#/c/20915/12/testdata/workloads/functional-query/queries/QueryTest/mysql-ext-jdbc-tables.test@185
PS12, Line 185: 2009-01-09
nit: just curious why the values of date columns in MySQL database are
different from Postgres and Impala?
--
To view, visit http://gerrit.cloudera.org:8080/20915
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibf13cbefaad812a0f78755c5791d82b24a3395e4
Gerrit-Change-Number: 20915
Gerrit-PatchSet: 12
Gerrit-Owner: gaurav singh
Gerrit-Reviewer: Abhishek Rawat
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Wenzhe Zhou
Gerrit-Reviewer: gaurav singh
Gerrit-Comment-Date: Thu, 01 Feb 2024 18:01:34 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20204 ) Change subject: IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile .. Patch Set 10: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/15134/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/20204 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee0b4967429ea733729ad8e44df32e3b24b88525 Gerrit-Change-Number: 20204 Gerrit-PatchSet: 10 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 01 Feb 2024 17:55:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12426: Query History Table
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20770 ) Change subject: IMPALA-12426: Query History Table .. Patch Set 17: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/15133/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/20770 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2d2da9d450fba4e789400cfa62927fc25d34f844 Gerrit-Change-Number: 20770 Gerrit-PatchSet: 17 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jason Fehr Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Thu, 01 Feb 2024 17:54:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile
Zoltan Borok-Nagy has uploaded a new patch set (#11) to the change originally created by Gergely Fürnstáhl. ( http://gerrit.cloudera.org:8080/20204 ) Change subject: IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile .. IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile Including the snapshot id of Iceberg tables for the Iceberg SCAN operators can be useful to enable replayable queries. Replayable queries are useful, so we can better investigate performance problems / bugs. Testing: - Updated planner tests, added e2e test for time travel Change-Id: Iee0b4967429ea733729ad8e44df32e3b24b88525 --- M fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java M fe/src/main/java/org/apache/impala/util/IcebergUtil.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java M fe/src/test/java/org/apache/impala/testutil/TestUtils.java M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-predicates-disabled-subsetting.test M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-predicates.test M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-v2-delete.test M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-v2-tables-hash-join.test M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-v2-tables.test M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-v2-update.test M testdata/workloads/functional-planner/queries/PlannerTest/insert-sort-by-zorder.test M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test M tests/query_test/test_iceberg.py 15 files changed, 509 insertions(+), 31 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/20204/11 -- To view, visit http://gerrit.cloudera.org:8080/20204 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iee0b4967429ea733729ad8e44df32e3b24b88525 Gerrit-Change-Number: 20204 Gerrit-PatchSet: 11 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20204 ) Change subject: IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile .. Patch Set 11: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10226/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/20204 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee0b4967429ea733729ad8e44df32e3b24b88525 Gerrit-Change-Number: 20204 Gerrit-PatchSet: 11 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 01 Feb 2024 17:47:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest
Hello Quanlong Huang, Abhishek Rawat, Wenzhe Zhou, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20922 to look at the new patch set (#5). Change subject: IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest .. IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest Querying against large-scale databases is a good way for testing Impala. However, it is impractical to do in a single-node development machine. Frontend testing does not run the test query in the backend executor and can benefit from simulated large-scale test cases. This patch attempts to do it by instrumenting the CatalogD metadata loading code to scale tpcds_partitioned_parquet_snap by injecting column stats from a 3TB TPC-DS dataset in TpcdsCpuCostPlannerTest. The large-scale column stats are expressed in stats-3TB.json, taken by running "SHOW COLUMN STATS" and "DESCRIBE FORMATTED" queries on a 3TB dataset loaded using impala-tpcds-kit. It is parsed and then piggy-backed through RuntimeEnv. Code that populates stats metadata (caller of FeCatalogUtils.getRowCount(), FeCatalogUtils.getTotalSize(), and FeCatalogUtils.injectColumnStats()) are instrumented to populate stats from RuntimeEnv instead of Metastore. Scaled-up tables are invalidated before a test run to reload them with new high-scale stats. This patch also adds a scan range limit injection to force ScanNode over a single file table to act as if it scans a multi-files table. tpcds_partitioned_schema_template.sql is modified to match column names and types from impala-tpcds-kit. The test files under PlannerTest/tpcds_cpu_cost/ is replaced with queries that are specifically generated to run against the 3TB scale factor of the TPC-DS dataset (https://github.com/cloudera/impala-tpcds-kit/blob/separate_queries_per_scale_factor/queries/sf3000/). All query plans match with query plans obtained through real query runs in a large cluster except for a few mismatches due to the hard limit on the number of files at a table. Below are 3 queries out of 103 that still do not have a matching shape and the reasons. +-+--+ | Q | Reason | +-+--+ | 10a | different num files in customer_demographics | | 34 | different num files in customer | | 69 | different num files in customer | +-+--+ Testing: - Scale tables of tpcds_partitioned_parquet_snap in TpcdsCpuCostPlannerTest to simulate 3TB TPC-DS. The number of executors is raised from 3 to 10, and REPLICA_PREFERENCE=REMOTE to ignore data locality. - Pass core tests. Change-Id: Iaffddd70c2da8376ca6c40f65606bbac46c34de7 --- M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java A fe/src/main/java/org/apache/impala/catalog/SideloadTableStats.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java M fe/src/main/java/org/apache/impala/catalog/local/LocalTable.java M fe/src/main/java/org/apache/impala/common/RuntimeEnv.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java M fe/src/test/java/org/apache/impala/planner/TpcdsCpuCostPlannerTest.java A fe/src/test/java/org/apache/impala/testutil/StatsJsonParser.java M testdata/datasets/tpcds_partitioned/tpcds_partitioned_schema_template.sql A testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/stats-3TB.json M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q01.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q02.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q03.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q04.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q05.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q06.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q07.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q08.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q09.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q10a.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q11.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q12.test M testdata/workloads/functional-planner/queries
[Impala-ASF-CR] IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20204 ) Change subject: IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile .. Patch Set 10: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10225/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/20204 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee0b4967429ea733729ad8e44df32e3b24b88525 Gerrit-Change-Number: 20204 Gerrit-PatchSet: 10 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 01 Feb 2024 17:34:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile
Zoltan Borok-Nagy has uploaded a new patch set (#10) to the change originally created by Gergely Fürnstáhl. ( http://gerrit.cloudera.org:8080/20204 ) Change subject: IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile .. IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile Including the snapshot id of Iceberg tables for the Iceberg SCAN operators can be useful to enable replayable queries. Replayable queries are useful, so we can better investigate performance problems / bugs. Testing: - Updated planner tests, added e2e test for time travel Change-Id: Iee0b4967429ea733729ad8e44df32e3b24b88525 --- M fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java M fe/src/main/java/org/apache/impala/util/IcebergUtil.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java M fe/src/test/java/org/apache/impala/testutil/TestUtils.java M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-predicates-disabled-subsetting.test M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-v2-delete.test M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-v2-tables-hash-join.test M testdata/workloads/functional-planner/queries/PlannerTest/iceberg-v2-tables.test M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test M tests/query_test/test_iceberg.py 12 files changed, 446 insertions(+), 26 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/20204/10 -- To view, visit http://gerrit.cloudera.org:8080/20204 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iee0b4967429ea733729ad8e44df32e3b24b88525 Gerrit-Change-Number: 20204 Gerrit-PatchSet: 10 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/20204 ) Change subject: IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile .. Patch Set 9: (3 comments) Thanks for the comments! http://gerrit.cloudera.org:8080/#/c/20204/9/fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java File fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java: http://gerrit.cloudera.org:8080/#/c/20204/9/fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java@74 PS9, Line 74: private long snapshotId_; > This could be final. Done http://gerrit.cloudera.org:8080/#/c/20204/9/fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java@277 PS9, Line 277: > nit: indentation Done http://gerrit.cloudera.org:8080/#/c/20204/9/fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java File fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java: http://gerrit.cloudera.org:8080/#/c/20204/9/fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java@145 PS9, Line 145: IcebergUtil.getSnapshotId(getIceTable(), tblRef_.getTimeTravelSpec()); > Shouldn't we put this inside the IcebergScanNode's constructor? I think we An IcebergScanPlanner can create up to 3 IcebergScanNode objects, so we can avoiding calling IcebergUtil.getSnapshotId() multiple times. The table ref of the DELETE scan node currently doesn't get a time travel spec, so we would also need to overload TableRef.newTableRef(). https://gerrit.cloudera.org/#/c/20866/ will add another overload to IcebergUtil.planFiles() that will take a 'long snapshotId', so we can also use 'snapshotId_' for it in filterFileDescriptors(). That being said, I don't have a too strong opinion about this, as the typical non-time travel case should be fairly efficient. Even in the case when we have time travel it shouldn't be too expensive, as we are only creating Scan objects without invoking planFiles() on them. -- To view, visit http://gerrit.cloudera.org:8080/20204 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee0b4967429ea733729ad8e44df32e3b24b88525 Gerrit-Change-Number: 20204 Gerrit-PatchSet: 9 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 01 Feb 2024 17:33:36 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/20922 ) Change subject: IMPALA-12726: Simulate large-scale query in TpcdsCpuCostPlannerTest .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/20922/4/fe/src/main/java/org/apache/impala/common/RuntimeEnv.java File fe/src/main/java/org/apache/impala/common/RuntimeEnv.java: http://gerrit.cloudera.org:8080/#/c/20922/4/fe/src/main/java/org/apache/impala/common/RuntimeEnv.java@39 PS4, Line 39: The value element is stored as Object to avoid referrencing : // SideloadTableStats class in org.apache.impala.common package. Looking around org/apache/impala/common, this package is pretty liberal in doing imports. There are imports from impala.analysis, impala.catalog, impala.thrift, and impala.util in common package. Maybe it is OK to import SideloadTableStats directly here. -- To view, visit http://gerrit.cloudera.org:8080/20922 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iaffddd70c2da8376ca6c40f65606bbac46c34de7 Gerrit-Change-Number: 20922 Gerrit-PatchSet: 4 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 01 Feb 2024 17:28:44 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12426: Query History Table
Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/20770 ) Change subject: IMPALA-12426: Query History Table .. Patch Set 16: (1 comment) http://gerrit.cloudera.org:8080/#/c/20770/16/be/src/service/workload-management.cc File be/src/service/workload-management.cc: http://gerrit.cloudera.org:8080/#/c/20770/16/be/src/service/workload-management.cc@635 PS16, Line 635: default: : break; > nit: optional: this can be made more robust by also handling other TCatalog I got a compiler error because I didn't handle every case from the enum. I like the idea of including every enum option in the switch and setting the default case to error. I implemented that technique. -- To view, visit http://gerrit.cloudera.org:8080/20770 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2d2da9d450fba4e789400cfa62927fc25d34f844 Gerrit-Change-Number: 20770 Gerrit-PatchSet: 16 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jason Fehr Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Thu, 01 Feb 2024 17:28:43 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12426: Query History Table
Hello Andrew Sherman, Riza Suminto, Michael Smith, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20770 to look at the new patch set (#17). Change subject: IMPALA-12426: Query History Table .. IMPALA-12426: Query History Table Adds the ability for users to specify that Impala will create and maintain an internal Iceberg table that contains data about all completed queries. This table is automatically created at startup by each coordinator if it does not exist. Then, most completed queries are queued in memory and flushed to the query history table at a set interval (either minutes or number of records). Set, use, and show queries are not written to this table. This commit leverages the InternalServer class to maintain the query history table. Ctest unit tests have been added to assert the various pieces of code. New custom cluster tests have been added to assert the query history table is properly populated with completed queries. Negative testing consists of attempting sql injection attacks and syntactically incorrect queries. Impala built-in string functions benchmarks have been updated to include the new built-in functions. Change-Id: I2d2da9d450fba4e789400cfa62927fc25d34f844 --- M be/src/benchmarks/expr-benchmark.cc M be/src/exprs/expr-test.cc M be/src/exprs/string-functions-ir.cc M be/src/exprs/string-functions.h M be/src/runtime/query-driver.cc M be/src/runtime/query-driver.h M be/src/scheduling/scheduler.cc M be/src/scheduling/scheduler.h M be/src/service/CMakeLists.txt M be/src/service/client-request-state.cc M be/src/service/client-request-state.h M be/src/service/impala-http-handler.cc M be/src/service/impala-http-handler.h M be/src/service/impala-server.cc M be/src/service/impala-server.h M be/src/service/internal-server-test.cc M be/src/service/internal-server.cc M be/src/service/internal-server.h A be/src/service/query-state-record.cc A be/src/service/query-state-record.h A be/src/service/workload-management.cc M be/src/util/CMakeLists.txt M be/src/util/backend-gflag-util.cc M be/src/util/impalad-metrics.cc M be/src/util/impalad-metrics.h A be/src/util/network-util-test.cc M be/src/util/network-util.h M be/src/util/runtime-profile.cc M be/src/util/runtime-profile.h M be/src/util/string-util-test.cc M be/src/util/string-util.cc M be/src/util/string-util.h A be/src/util/ticker.h M be/src/util/uid-util-test.cc M be/src/util/uid-util.h M bin/run_clang_tidy.sh M common/function-registry/impala_functions.py M common/thrift/BackendGflags.thrift M common/thrift/metrics.json M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/util/CatalogBlacklistUtils.java M tests/beeswax/impala_beeswax.py M tests/common/custom_cluster_test_suite.py A tests/custom_cluster/test_query_log.py A tests/util/assert_time.py A tests/util/memory.py A tests/util/retry.py 48 files changed, 3,445 insertions(+), 388 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/70/20770/17 -- To view, visit http://gerrit.cloudera.org:8080/20770 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2d2da9d450fba4e789400cfa62927fc25d34f844 Gerrit-Change-Number: 20770 Gerrit-PatchSet: 17 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jason Fehr Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Riza Suminto
[Impala-ASF-CR] IMPALA-12503: Support date data type for predicates for external data source table
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20915 ) Change subject: IMPALA-12503: Support date data type for predicates for external data source table .. Patch Set 12: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/15132/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20915 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibf13cbefaad812a0f78755c5791d82b24a3395e4 Gerrit-Change-Number: 20915 Gerrit-PatchSet: 12 Gerrit-Owner: gaurav singh Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: gaurav singh Gerrit-Comment-Date: Thu, 01 Feb 2024 15:55:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12503: Support date data type for predicates for external data source table
gaurav singh has posted comments on this change. ( http://gerrit.cloudera.org:8080/20915 ) Change subject: IMPALA-12503: Support date data type for predicates for external data source table .. Patch Set 12: (1 comment) http://gerrit.cloudera.org:8080/#/c/20915/11/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java File java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java: http://gerrit.cloudera.org:8080/#/c/20915/11/java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java@220 PS11, Line 220: UTC > should be "UTC", Coordinated Universal Time. Done -- To view, visit http://gerrit.cloudera.org:8080/20915 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibf13cbefaad812a0f78755c5791d82b24a3395e4 Gerrit-Change-Number: 20915 Gerrit-PatchSet: 12 Gerrit-Owner: gaurav singh Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: gaurav singh Gerrit-Comment-Date: Thu, 01 Feb 2024 15:30:15 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12503: Support date data type for predicates for external data source table
gaurav singh has posted comments on this change. ( http://gerrit.cloudera.org:8080/20915 ) Change subject: IMPALA-12503: Support date data type for predicates for external data source table .. Patch Set 12: (2 comments) http://gerrit.cloudera.org:8080/#/c/20915/11//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/20915/11//COMMIT_MSG@15 PS11, Line 15: '=', '>', '<', '>=', '<=' > Please add more tests for operators '>=', '<=', '!=', 'between' Done http://gerrit.cloudera.org:8080/#/c/20915/11//COMMIT_MSG@18 PS11, Line 18: > nit: this was removed Done -- To view, visit http://gerrit.cloudera.org:8080/20915 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibf13cbefaad812a0f78755c5791d82b24a3395e4 Gerrit-Change-Number: 20915 Gerrit-PatchSet: 12 Gerrit-Owner: gaurav singh Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: gaurav singh Gerrit-Comment-Date: Thu, 01 Feb 2024 15:30:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12503: Support date data type for predicates for external data source table
Hello Abhishek Rawat, Wenzhe Zhou, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20915 to look at the new patch set (#12). Change subject: IMPALA-12503: Support date data type for predicates for external data source table .. IMPALA-12503: Support date data type for predicates for external data source table This patch adds support for datatype date as predicates for external data sources. Testing: - Added tests for binary date predicates with operators: '=', '>', '<', '>=', '<=', '!=', 'BETWEEN'. Change-Id: Ibf13cbefaad812a0f78755c5791d82b24a3395e4 --- M fe/src/main/java/org/apache/impala/planner/DataSourceScanNode.java M java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/JdbcDataSource.java M java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/DatabaseAccessor.java M java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/dao/GenericJdbcDatabaseAccessor.java M java/ext-data-source/jdbc/src/main/java/org/apache/impala/extdatasource/jdbc/util/QueryConditionUtil.java M testdata/bin/create-ext-data-source-table.sql M testdata/bin/load-ext-data-sources.sh M testdata/bin/setup-mysql-env.sh A testdata/workloads/functional-query/queries/QueryTest/impala-ext-jdbc-tables-predicates.test M testdata/workloads/functional-query/queries/QueryTest/jdbc-data-source.test M testdata/workloads/functional-query/queries/QueryTest/mysql-ext-jdbc-tables.test M tests/custom_cluster/test_ext_data_sources.py 12 files changed, 463 insertions(+), 57 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/15/20915/12 -- To view, visit http://gerrit.cloudera.org:8080/20915 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibf13cbefaad812a0f78755c5791d82b24a3395e4 Gerrit-Change-Number: 20915 Gerrit-PatchSet: 12 Gerrit-Owner: gaurav singh Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: gaurav singh
[Impala-ASF-CR] IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/20204 ) Change subject: IMPALA-12072: Include snapshot id of Iceberg tables in query plan / profile .. Patch Set 9: (3 comments) Thanks for reviving this change Zoltan. Could you check if my idea to simplify the code makes sense? http://gerrit.cloudera.org:8080/#/c/20204/9/fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java File fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java: http://gerrit.cloudera.org:8080/#/c/20204/9/fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java@74 PS9, Line 74: private long snapshotId_; This could be final. http://gerrit.cloudera.org:8080/#/c/20204/9/fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java@277 PS9, Line 277: nit: indentation http://gerrit.cloudera.org:8080/#/c/20204/9/fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java File fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java: http://gerrit.cloudera.org:8080/#/c/20204/9/fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java@145 PS9, Line 145: IcebergUtil.getSnapshotId(getIceTable(), tblRef_.getTimeTravelSpec()); Shouldn't we put this inside the IcebergScanNode's constructor? I think we could avoid passing it to every object because the TableRef is passed to the ScanNode, which holds a reference to the IcebergTable object. We might not even need the getSnapshotId() method. -- To view, visit http://gerrit.cloudera.org:8080/20204 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee0b4967429ea733729ad8e44df32e3b24b88525 Gerrit-Change-Number: 20204 Gerrit-PatchSet: 9 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 01 Feb 2024 11:39:03 + Gerrit-HasComments: Yes
