[
https://issues.apache.org/jira/browse/IMPALA-14811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18069401#comment-18069401
]
ASF subversion and git services commented on IMPALA-14811:
----------------------------------------------------------
Commit bacd1a561c89465a15d78c9d220e758443e53790 in impala's branch
refs/heads/master from Steve Carlin
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=bacd1a561 ]
IMPALA-14820: Calcite Planner: Memory allocations are too high
The following queries are showing memory allocations that are too
high:
In bloom_filters.test:
with l as (select * from tpch.lineitem UNION ALL select * from tpch.lineitem)
select STRAIGHT_JOIN count(*) from (select * from tpch.lineitem a LIMIT 1) a
join (select * from l LIMIT 125000) b on a.l_orderkey = -b.l_orderkey
In admission-max-min-mem-limits.test:
select * from functional_parquet.alltypes limit 1;
The issue in the first query is that the stats need to be recomputed after
the limit is set. Setting the limit propagates into its children nodes, so
the stats also need to be recomputed when the limit is set.
The issue in the second query is that the top level needs to know when there
is only one row output, and the case with a "limit 1" clause was missing from
the check. Note: Calcite uses the Sort RelNode for "order by", "limit" and
"offset".
The bloom_filters.test is still incorrect, but the memory allocation is much
lower after this commit. IMPALA-14811 has been created for this issue.
Change-Id: I9e84061b65f115ea65a9d2c2d40173843040ca0c
Reviewed-on: http://gerrit.cloudera.org:8080/24079
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Michael Smith <[email protected]>
Reviewed-by: Quanlong Huang <[email protected]>
> Calcite Planner: Unneeded SelectNode created below JoinNode
> -----------------------------------------------------------
>
> Key: IMPALA-14811
> URL: https://issues.apache.org/jira/browse/IMPALA-14811
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Steve Carlin
> Priority: Major
>
> In bloom_filters.test for the query:
>
> {code:java}
> with l as (select * from tpch.lineitem UNION ALL select * from tpch.lineitem)
> select STRAIGHT_JOIN count(*) from (select * from tpch.lineitem a LIMIT 1) a
> join (select * from l LIMIT 125000) b on a.l_orderkey = -b.l_orderkey;
> {code}
> ...an unnecessary SelectNode is created underneath the Join.
> We should create a rule that removes the LogicalFilter under a LogicalJoin if:
> a) the join is an inner join
> b) There is a filter directly underneath the join
> c) the only condition in the filter is an is not null clause
> d) the join is either an inner join or, if it's a left/right join, the
> left/right side contains the filter.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
