[
https://issues.apache.org/jira/browse/IMPALA-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18069400#comment-18069400
]
ASF subversion and git services commented on IMPALA-14820:
----------------------------------------------------------
Commit bacd1a561c89465a15d78c9d220e758443e53790 in impala's branch
refs/heads/master from Steve Carlin
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=bacd1a561 ]
IMPALA-14820: Calcite Planner: Memory allocations are too high
The following queries are showing memory allocations that are too
high:
In bloom_filters.test:
with l as (select * from tpch.lineitem UNION ALL select * from tpch.lineitem)
select STRAIGHT_JOIN count(*) from (select * from tpch.lineitem a LIMIT 1) a
join (select * from l LIMIT 125000) b on a.l_orderkey = -b.l_orderkey
In admission-max-min-mem-limits.test:
select * from functional_parquet.alltypes limit 1;
The issue in the first query is that the stats need to be recomputed after
the limit is set. Setting the limit propagates into its children nodes, so
the stats also need to be recomputed when the limit is set.
The issue in the second query is that the top level needs to know when there
is only one row output, and the case with a "limit 1" clause was missing from
the check. Note: Calcite uses the Sort RelNode for "order by", "limit" and
"offset".
The bloom_filters.test is still incorrect, but the memory allocation is much
lower after this commit. IMPALA-14811 has been created for this issue.
Change-Id: I9e84061b65f115ea65a9d2c2d40173843040ca0c
Reviewed-on: http://gerrit.cloudera.org:8080/24079
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Michael Smith <[email protected]>
Reviewed-by: Quanlong Huang <[email protected]>
> Calcite planner: Memory allocations too high for various queries
> ----------------------------------------------------------------
>
> Key: IMPALA-14820
> URL: https://issues.apache.org/jira/browse/IMPALA-14820
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Steve Carlin
> Priority: Major
>
> The following two queries have bad memory calculations and the tests are
> failing:
> In admission-max-min-mem-limits.test
> select * from functional_parquet.alltypes limit 1;
> In bloom_filters.test
> with l as (select * from tpch.lineitem UNION ALL select * from tpch.lineitem)
> select STRAIGHT_JOIN count(*) from (select * from tpch.lineitem a LIMIT 1) a
> join (select * from l LIMIT 125000) b on a.l_orderkey = -b.l_orderkey;
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
