[
https://issues.apache.org/jira/browse/IMPALA-11008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517281#comment-17517281
]
ASF subversion and git services commented on IMPALA-11008:
----------------------------------------------------------
Commit abfa5a72b6a6046037f27ad02d19b31f7b31c4e5 in impala's branch
refs/heads/master from xqhe
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=abfa5a72b ]
IMPALA-11008: fix incorrect to propagate inferred predicates
It is incorrect to propagate predicates inferred from equi-join
conjuncts into a plan subtree that is on the nullable side of an
outer join if the predicate is not null-filtering for the nullable
side.
For example:
SELECT *
FROM (
SELECT id IS NOT NULL
AND col IS NULL AS a
FROM (
SELECT A.id, B.col
FROM A
LEFT JOIN B ON A.id = B.id
) t
) t
WHERE a = 1
Before this patch the inferred predicate '(B.id is not null and
B.col is null) = 1' is evaluated at the scanner of B. This is
incorrect since the predicate '(A.id is not null and B.col is null)
= 1' is not null-filtering for B.
To generate the inferred predicate we substitue the non-outer-join
slots first and use 'isNullableConjunct' to do a more strict check
on the conjunct before the final substitution.
Tests:
- Add plan tests in predicate-propagation.test
- Add new query tests to verify the correctness of inferred
predicates propagation
- Ran the full set of verifications in Impala Public Jenkins
Change-Id: I9e64230f6d0c2b9ef1560186ceba349a5920ccdf
Reviewed-on: http://gerrit.cloudera.org:8080/18234
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Invalid to propagate inferred predicates into the nullable side of an outer
> join
> --------------------------------------------------------------------------------
>
> Key: IMPALA-11008
> URL: https://issues.apache.org/jira/browse/IMPALA-11008
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.1.0, Impala 3.2.0, Impala 4.0.0, Impala 3.4.0
> Reporter: Xianqing He
> Assignee: Xianqing He
> Priority: Major
> Labels: correctness
>
> {code:java}
> Query: explain select * from (
> select case when id is not null and some_nulls is null then true else null
> end a from (select straight_join t1.id, t2.some_nulls from
> functional.nullrows t1 left join functional.nullrows t2 on t1.id=t2.id) t)t
> where a is not null
> Max Per-Host Resource Reservation: Memory=1.95MB Threads=3
> Per-Host Resource Estimates: Memory=66MB
> Codegen disabled by planner
> ""
> PLAN-ROOT SINK
> |
> 02:HASH JOIN [LEFT OUTER JOIN]
> | hash predicates: t1.id = t2.id
> | other predicates: CASE WHEN t1.id IS NOT NULL AND t2.some_nulls IS NULL
> THEN TRUE ELSE NULL END IS NOT NULL
> | row-size=39B cardinality=26
> |
> |--01:SCAN HDFS [functional.nullrows t2]
> | partitions=1/1 files=1 size=541B
> | predicates: CASE WHEN t2.id IS NOT NULL AND t2.some_nulls IS NULL THEN
> TRUE ELSE NULL END IS NOT NULL
> | row-size=26B cardinality=3
> |
> 00:SCAN HDFS [functional.nullrows t1]
> partitions=1/1 files=1 size=541B
> row-size=13B cardinality=26 {code}
> It is incorrect to propagate this inferred predicates into a plan subtree
> that is on the nullable side of an outer join.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
