pengdou has uploaded a new patch set (#5). (
http://gerrit.cloudera.org:8080/17830 )
Change subject: IMPALA-10902: In the case of outer join connections, skip
unnecessary BE check
......................................................................
IMPALA-10902: In the case of outer join connections, skip unnecessary BE check
Function org.apache.impala.analysis.Analyzer#isTrueWithNullSlots is
called in a variety of scenarios, for example outer join, and
the function finally call expensive BE Native function
org.apache.impala.service.FeSupport#NativeEvalExprsWithoutRow.
If sql contains hundreds of outer join and the predicate is an
inPredicate with thousands of literal child, the BE Native function
will be called hundreds of times, cause create single node plan quite
slow, for example the below query compilation took 6s449ms, among
them, create single node plan took 5s166ms.
```sql
select
t.id, t.bool_col_1, t.bool_col_2, t.bool_col_3, ..., t.bool_col_100
from
(select
t1.id id , t1.bool_col bool_col_1, t2.bool_col bool_col_2,
t3.bool_col bool_col_3,..., t100.bool_col bool_col_100
from
(select id, bool_col from functional.alltypessmall) t1
left outer join
(select id, bool_col from functional.alltypessmall) t2
on t1.id = t2.id
left outer join
(select id, bool_col from functional.alltypessmall) t3
on t1.id = t3.id
...
left outer join
(select id, bool_col from functional.alltypessmall) t100
on t1.id = t100.id
) t
where
id in (
1,2,3,..., 5000
);
```
```text
Query Compilation: 6s449ms
- Single node plan created: 5s283ms (5s166ms)
```
To solve the problem, we do fe evaluate when we meet simple predicate
as below:
1. IsNullPredicate with single slotRef
2. BinaryPredicate with single slotRef lhs and literal rhs
3. InPredicate with single slotRef and literal bounds
4. CompoundPredicate and each child satisfy 1,2,3
With the patch, query compilation took 1s728ms, among them, create
single node plan took 466.353ms.
```text
Query Compilation: 1s728ms
- Single node plan created: 748.258ms (466.353ms)
```
Change-Id: If6f4db9b5aa00ee5117118696c58c07ffaaf0b66
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java
2 files changed, 346 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/17830/5
--
To view, visit http://gerrit.cloudera.org:8080/17830
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If6f4db9b5aa00ee5117118696c58c07ffaaf0b66
Gerrit-Change-Number: 17830
Gerrit-PatchSet: 5
Gerrit-Owner: pengdou <pengdou1...@126.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Jian Zhang <zjsar...@gmail.com>
Gerrit-Reviewer: Xianqing He <hexianqing...@126.com>
