Tim Armstrong has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/16622 )
Change subject: IMPALA-10252: fix invalid runtime filters for outer joins ...................................................................... IMPALA-10252: fix invalid runtime filters for outer joins The planner generates runtime filters for non-join conjuncts assigned to LEFT OUTER and FULL OUTER JOIN nodes. This is correct in many cases where NULLs stemming from unmatched rows would result in the predicate evaluating to false. E.g. x = y is always false is y is NULL. However, it is incorrect if the NULL returned from the unmatched row can result in the predicate evaluating to true. E.g. x = isnull(y, 1) can return true even if y is NULL. The fix is to detect cases when the source expression from the left input of the join returns non-NULL for null inputs and then skip generating the filter. Testing: Added regression tests: * Planner tests for LEFT OUTER and FULL OUTER where the runtime filter was incorrectly generated before this patch. * Enabled end-to-end test that was previously failing. * Added a new runtime filter test that will execute on both Parquet and Kudu (which are subtly different because of nullability of slots). Ran exhaustive tests. Change-Id: I507af1cc8df15bca21e0d8555019997812087261 --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java M testdata/workloads/functional-planner/queries/PlannerTest/runtime-filter-propagation.test M testdata/workloads/functional-query/queries/QueryTest/runtime_filters.test M testdata/workloads/functional-query/queries/QueryTest/subquery.test 5 files changed, 144 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/16622/2 -- To view, visit http://gerrit.cloudera.org:8080/16622 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I507af1cc8df15bca21e0d8555019997812087261 Gerrit-Change-Number: 16622 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com>
