Jian Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18862 )
Change subject: IMPALA-11485: Pushdown LIMIT through UNION ALL and LEFT/RIGHT OUTER JOIN ...................................................................... Patch Set 7: (2 comments) http://gerrit.cloudera.org:8080/#/c/18862/7/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java File fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java: http://gerrit.cloudera.org:8080/#/c/18862/7/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java@370 PS7, Line 370: private static void checkAndApplyLimitPushdown(PlanNode root, Maybe a better implementation can be this: void applyLimitPushdownOptimization(PlanNode cur, long limit, ...) { if (cur instance of SortNode || cur instance of AggregationNode || cur instance of SubplanNode || cur instance of AnalyticEvalNode) { // stop finding nodes can pushdown limit when we meet pipeline breakers return } else if (cur instanceof UnionNode) { // pushdown limit to union all node cur.setLimit(limit); cur.computeStats(analyzer); // try to apply limit pushdown to every child for (child : cur.ChildNodes()) { applyLimitPushdownOptimization(child, limit, ...) } } else if (cur instance of JoinNode) { // pushdown limit to the outer side of join node outerNode = cur.GetOuterNode(); outerNode.setLimit(limit); outerNode.computeStats(analyzer); // try to apply limit pushdown only to the outer side child applyLimitPushdownOptimization(outerNode, limit, ...) } else { // try to apply limit pushdown to every child of non pipeline breaker for (child : cur.ChildNodes()) { applyLimitPushdownOptimization(child, limit, ...) } } } http://gerrit.cloudera.org:8080/#/c/18862/7/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java@409 PS7, Line 409: root instanceof JoinNode if we stop finding nodes can be pushdown in JoinNode, the following scenario cannot be optimized: select * from ( select * from t1 unioin all select * from t2 ) t left join s on t.id=s.id limit 1; the expected optimized equivalent is: select * from ( select * from t1 limit 1 unioin all select * from t2 limit 1 limit 1 ) t left join s on t.id=s.id limit 1; -- To view, visit http://gerrit.cloudera.org:8080/18862 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia5d040c0a98e60639d7ce4b25ecf07a859c8a32c Gerrit-Change-Number: 18862 Gerrit-PatchSet: 7 Gerrit-Owner: Baike Xia <xiaba...@163.com> Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: Baike Xia <xiaba...@163.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Jian Zhang <zjsar...@gmail.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Comment-Date: Wed, 12 Oct 2022 09:38:25 +0000 Gerrit-HasComments: Yes
