Xianqing He has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/15047 )
Change subject: IMPALA-8361: Propagate predicates of outer-joined InlineView ...................................................................... IMPALA-8361: Propagate predicates of outer-joined InlineView This is an improvement that tries to propagate predicates of the nullable side of the outer join into inline view. For example: SELECT * FROM functional.alltypessmall a LEFT JOIN ( SELECT id, upper(string_col) AS upper_val, length(string_col) AS len FROM functional.alltypestiny ) b ON a.id = b.id WHERE b.upper_val is NULL and b.len = 0 Before this change, the predicate b.len=0 can't be migrated into inline view since that is on the nullable side of an outer join if the predicate evaluates in the inline view nulls will not be rejected. However,we can be more aggressive. In particular, the predicates that must be evaluted at a join node but can also be safely evaluted by the outer-joined inline view. Such predicates are not marked as assigned. The predicates propagate into the inline view and also be evaluated at a join node. We can divide predicates into two types.One that satisfies the condition that same as Analyzer#canEvalPredicate can be migrated into inline view, and one that satisfies the below three conditions is safe to be propagated into the nullable side of an outer join. 1) The predicate needs to be bound by tupleIds. 2) The predicate is not on-clause. 3) The predicate evaluates to false when all its referenced tuples are NULL. Therefore, 'b.upper_val is NULL' cannot be propagated to inline view but ‘b.len = 0’ can be propagated to inline view. Tests: * Add plan tests in inline-view.test * One baseline plan in inline-view.test, one in nested-collections.test and two in predicate-propagation.test had to be updated * Ran the full set of verifications in Impala Public Jenkins Change-Id: I6c23a45aeb5dd1aa06a95c9aa8628ecbe37ef2c1 --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/MultiAggregateInfo.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M testdata/workloads/functional-planner/queries/PlannerTest/inline-view.test M testdata/workloads/functional-planner/queries/PlannerTest/nested-collections.test M testdata/workloads/functional-planner/queries/PlannerTest/predicate-propagation.test 6 files changed, 334 insertions(+), 57 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/47/15047/9 -- To view, visit http://gerrit.cloudera.org:8080/15047 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6c23a45aeb5dd1aa06a95c9aa8628ecbe37ef2c1 Gerrit-Change-Number: 15047 Gerrit-PatchSet: 9 Gerrit-Owner: Xianqing He <hexianqing...@126.com> Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Xianqing He <hexianqing...@126.com>