Xianqing He has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/15047 )
Change subject: IMPALA-8361: Propagate predicates of outer-joined InlineView ...................................................................... IMPALA-8361: Propagate predicates of outer-joined InlineView This is improvement that propagate predicates of the nullable side of the outer join into inline view. I refactored the code "Analyzer#canEvalPredicate()" and add "SingleNodePlanner#getConjunctsToInlineView()" to get conjucts can be correctly to migrate or propagate into inline view. Before this change, we skip predicate b.upper_val='1' since canEvalPredicate() returns false on it. Because it doesn't satisfy isLastOjMaterializedByTupleIds(). So we can't migrate it inside the inline view. However, we can be more aggressive. It's correct to duplicate(not migrate) this predicate inside the inline view since it's not evaluted to true with null slots. This patch picks up the predicates that being skipped due to not satisfying isLastOjMaterializedByTupleIds() and duplicates them if they satisfy !isTrueWithNullSlots(). For example: EXPLAIN SELECT count(*) FROM functional.alltypessmall a LEFT JOIN (SELECT id, upper(string_col) AS upper_val FROM functional.alltypestiny) b ON a.id=b.id WHERE b.upper_val='1'; +------------------------------------------------------------+ | Explain String | +------------------------------------------------------------+ | Max Per-Host Resource Reservation: Memory=1.95MB Threads=5 | | Per-Host Resource Estimates: Memory=86MB | | Codegen disabled by planner | | | | PLAN-ROOT SINK | | | | | 06:AGGREGATE [FINALIZE] | | | output: count:merge(*) | | | row-size=8B cardinality=1 | | | | | 05:EXCHANGE [UNPARTITIONED] | | | | | 03:AGGREGATE | | | output: count(*) | | | row-size=8B cardinality=1 | | | | | 02:HASH JOIN [LEFT OUTER JOIN, BROADCAST] | | | hash predicates: a.id = id | | | other predicates: upper(string_col) = '1' <-eval after join | | row-size=21B cardinality=100 | | | | | |--04:EXCHANGE [BROADCAST] | | | | | | | 01:SCAN HDFS [functional.alltypestiny] | | | HDFS partitions=4/4 files=4 size=460B | | | predicates: upper(string_col) = '1' <-push down | | | row-size=17B cardinality=4 | | | | | 00:SCAN HDFS [functional.alltypessmall a] | | HDFS partitions=4/4 files=4 size=6.32KB | | row-size=4B cardinality=100 | +------------------------------------------------------------+ Change-Id: I6c23a45aeb5dd1aa06a95c9aa8628ecbe37ef2c1 --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/MultiAggregateInfo.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M testdata/workloads/functional-planner/queries/PlannerTest/inline-view.test M testdata/workloads/functional-planner/queries/PlannerTest/nested-collections.test M testdata/workloads/functional-planner/queries/PlannerTest/predicate-propagation.test 6 files changed, 146 insertions(+), 57 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/47/15047/5 -- To view, visit http://gerrit.cloudera.org:8080/15047 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6c23a45aeb5dd1aa06a95c9aa8628ecbe37ef2c1 Gerrit-Change-Number: 15047 Gerrit-PatchSet: 5 Gerrit-Owner: Xianqing He <hexianqing...@126.com> Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Xianqing He <hexianqing...@126.com>