Steve Carlin has uploaded this change for review. (
http://gerrit.cloudera.org:8080/24079
Change subject: IMPALA-14820: Calcite Planner: Memory allocations are too high
......................................................................
IMPALA-14820: Calcite Planner: Memory allocations are too high
The following queries are showing memory allocations that are too
high:
In bloom_filters.test:
with l as (select * from tpch.lineitem UNION ALL select * from tpch.lineitem)
select STRAIGHT_JOIN count(*) from (select * from tpch.lineitem a LIMIT 1) a
join (select * from l LIMIT 125000) b on a.l_orderkey = -b.l_orderkey
In admission-max-min-mem-limits.test:
select * from functional_parquet.alltypes limit 1;
The issue in the first query is that the stats need to be recomputed after
the limit is set. Setting the limit propagates into its children nodes, so
the stats also need to be recomputed when the limit is set.
The issue in the second query is that the top level needs to know when there
is only one row output, and the case with a "limit 1" clause was missing from
the check.
The bloom_filters.test is still incorrect, but the memory allocation is much
lower after this commit. IMPALA-14811 has been created for this issue.
Change-Id: I9e84061b65f115ea65a9d2c2d40173843040ca0c
---
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaSortRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteSingleNodePlanner.java
2 files changed, 16 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/79/24079/2
--
To view, visit http://gerrit.cloudera.org:8080/24079
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I9e84061b65f115ea65a9d2c2d40173843040ca0c
Gerrit-Change-Number: 24079
Gerrit-PatchSet: 2
Gerrit-Owner: Steve Carlin <[email protected]>
Gerrit-Reviewer: Aman Sinha <[email protected]>
Gerrit-Reviewer: Fang-Yu Rao <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
