[Impala-ASF-CR] IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19422 ) Change subject: IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown .. Patch Set 1: Thank Aman! Merging this. -- To view, visit http://gerrit.cloudera.org:8080/19422 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I26d6bd58be58d09a29b8b81972e76665f41cf103 Gerrit-Change-Number: 19422 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 17 Jan 2023 02:38:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown
Quanlong Huang has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19422 ) Change subject: IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown .. IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown When finding analytic conjuncts for analytic limit pushdown, the following conditions are checked: - The conjunct should be a binary predicate - Left hand side is a SlotRef referencing the analytic expression, e.g. "rn" of "row_number() as rn" - The underlying analytic function is rank(), dense_rank() or row_number() - The window frame is UNBOUNDED PRECEDING to CURRENT ROW - Right hand side is a valid numeric limit - The op is =, <, or <= See more details in AnalyticPlanner.inferPartitionLimits(). While checking the 2nd and 3rd condition, we get the source exprs of the SlotRef. The source exprs could be empty if the SlotRef is actually referencing a column of the table, i.e. a column materialized by the scan node. Currently, we check the first source expr directly regardless whether the list is empty, which causes the IndexOutOfBoundsException. This patch fixes it by augmenting the check to consider an empty list. Also fixes a similar code in AnalyticEvalNode. Tests: - Add FE and e2e regression tests Change-Id: I26d6bd58be58d09a29b8b81972e76665f41cf103 Reviewed-on: http://gerrit.cloudera.org:8080/19422 Reviewed-by: Aman Sinha Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java M fe/src/main/java/org/apache/impala/planner/AnalyticPlanner.java M testdata/workloads/functional-planner/queries/PlannerTest/limit-pushdown-partitioned-top-n.test M testdata/workloads/functional-query/queries/QueryTest/partitioned-top-n.test 4 files changed, 46 insertions(+), 2 deletions(-) Approvals: Aman Sinha: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/19422 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I26d6bd58be58d09a29b8b81972e76665f41cf103 Gerrit-Change-Number: 19422 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19422 ) Change subject: IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/19422 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I26d6bd58be58d09a29b8b81972e76665f41cf103 Gerrit-Change-Number: 19422 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Tue, 17 Jan 2023 01:38:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19422 ) Change subject: IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8968/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/19422 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I26d6bd58be58d09a29b8b81972e76665f41cf103 Gerrit-Change-Number: 19422 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Mon, 16 Jan 2023 20:31:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/19422 ) Change subject: IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown .. Patch Set 1: Code-Review+2 LGTM. Thanks for the patch. I am surprised we did not already have such a test case with mixed conjuncts. -- To view, visit http://gerrit.cloudera.org:8080/19422 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I26d6bd58be58d09a29b8b81972e76665f41cf103 Gerrit-Change-Number: 19422 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Mon, 16 Jan 2023 19:27:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19422 ) Change subject: IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12170/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19422 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I26d6bd58be58d09a29b8b81972e76665f41cf103 Gerrit-Change-Number: 19422 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Mon, 16 Jan 2023 12:00:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19422 Change subject: IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown .. IMPALA-11843: Fix IndexOutOfBoundsException in analytic limit pushdown When finding analytic conjuncts for analytic limit pushdown, the following conditions are checked: - The conjunct should be a binary predicate - Left hand side is a SlotRef referencing the analytic expression, e.g. "rn" of "row_number() as rn" - The underlying analytic function is rank(), dense_rank() or row_number() - The window frame is UNBOUNDED PRECEDING to CURRENT ROW - Right hand side is a valid numeric limit - The op is =, <, or <= See more details in AnalyticPlanner.inferPartitionLimits(). While checking the 2nd and 3rd condition, we get the source exprs of the SlotRef. The source exprs could be empty if the SlotRef is actually referencing a column of the table, i.e. a column materialized by the scan node. Currently, we check the first source expr directly regardless whether the list is empty, which causes the IndexOutOfBoundsException. This patch fixes it by augmenting the check to consider an empty list. Also fixes a similar code in AnalyticEvalNode. Tests: - Add FE and e2e regression tests Change-Id: I26d6bd58be58d09a29b8b81972e76665f41cf103 --- M fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java M fe/src/main/java/org/apache/impala/planner/AnalyticPlanner.java M testdata/workloads/functional-planner/queries/PlannerTest/limit-pushdown-partitioned-top-n.test M testdata/workloads/functional-query/queries/QueryTest/partitioned-top-n.test 4 files changed, 46 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19422/1 -- To view, visit http://gerrit.cloudera.org:8080/19422 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I26d6bd58be58d09a29b8b81972e76665f41cf103 Gerrit-Change-Number: 19422 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang
