[
https://issues.apache.org/jira/browse/CALCITE-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17034717#comment-17034717
]
Haisheng Yuan commented on CALCITE-3784:
----------------------------------------
[~julianhyde] It is not surprising large rexcalls consume gigabytes of memory.
A RexCall can be simplified to another different RexCall, its digest changes,
parent RexNode and RelNode will have different digest. When doing project
transpose or column pruning, column index might be changed, new RexCalls and
digests are generated for them. When the RelNode with this RexCall matched with
some rules, the rule match is added to RuleQueue, the rule match digest will
contain these Rel digests again (in older version). It can easily blow up the
memory.
> RexCall toString digest gives OOM while huge expression is evaluated
> --------------------------------------------------------------------
>
> Key: CALCITE-3784
> URL: https://issues.apache.org/jira/browse/CALCITE-3784
> Project: Calcite
> Issue Type: Bug
> Components: core
> Reporter: Ravi Kapoor
> Priority: Critical
>
> I have some complex query which has tens of thousands of rex expressions and
> this expression is used in filter expression in the query.
> On creating a filter below code gets called:
> {code}
> public RelBuilder filter(Iterable<CorrelationId> variablesSet,
> Iterable<? extends RexNode> predicates) {
> final RexNode simplifiedPredicates =
> simplifier.simplifyFilterPredicates(predicates);
> if (simplifiedPredicates == null) {
> return empty();
> }
> {code}
> RexSimplify further adds the rexnode in the Set<RexNode> calling hashcode()
> internally which calls toString()
> Is there any way to avoid this computeDigest Call which creates complex
> string object and blows up the RAM about 14GB?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
