[ https://issues.apache.org/jira/browse/CALCITE-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013494#comment-17013494 ]
Vladimir Sitnikov commented on CALCITE-2223: -------------------------------------------- 1) Recently I've merged two normalizations: "normalize rex", "use field types only (skip names) when comparing relation equivalence", and "simplify lossless casts". It looks like "simplify lossless cast" alone would heal the test case in the description of the issue, however, others might reduce planning space as well. 2) The conversion to appropriate conventions reduces planning space as well as it avoids producing irrelevant logical-enumerable based filters. In other words, if it wants to merge EnumerableFilter(EnumerableFiler(...)), then it would produce EnumerableFilter(input=enumerable) rather than input=logical. I'm not sure it does help a lot here, however, it was required to add assertions to Enumerable constructors that validate the inputs are enumerable. 3) https://github.com/apache/calcite/pull/1745 <-- this might help for your case as well. It turns out sometimes rules are never executed under the current RuleQueue implementation. PR#1745 fixes that by refraining from selecting rules that produce lots of relations. > ProjectMergeRule is infinitely matched when is applied after > ProjectReduceExpressionsRule > ----------------------------------------------------------------------------------------- > > Key: CALCITE-2223 > URL: https://issues.apache.org/jira/browse/CALCITE-2223 > Project: Calcite > Issue Type: Bug > Reporter: Vova Vysotskyi > Priority: Critical > Labels: pull-request-available > Attachments: > TestLimitWithExchanges_testPushLimitPastUnionExchange.png, heap_overview.png, > provenance_contents.png > > > For queries like this: > {code:sql} > select t1.f from (select cast(f as int) f, f from (select cast(f as int) f > from (values('1')) t(f))) as t1 > {code} > OOM is thrown when {{ProjectMergeRule}} is applied before applying > {{ProjectReduceExpressionsRule}} in VolcanoPlanner. > A simple test to reproduce this issue (in {{RelOptRulesTest}}): > {code:java} > @Test public void testOomProjectMergeRule() { > RelBuilder relBuilder = > RelBuilder.create(RelBuilderTest.config().build()); > RelNode relNode = relBuilder > .values(new String[]{"f"}, "1") > .project( > relBuilder.alias( > relBuilder.cast(relBuilder.field(0), SqlTypeName.INTEGER), > "f")) > .project( > relBuilder.alias( > relBuilder.cast(relBuilder.field(0), SqlTypeName.INTEGER), > "f0"), > relBuilder.alias(relBuilder.field(0), "f")) > .project( > relBuilder.alias(relBuilder.field(0), "f")) > .build(); > RelOptPlanner planner = relNode.getCluster().getPlanner(); > RuleSet ruleSet = > RuleSets.ofList( > ReduceExpressionsRule.PROJECT_INSTANCE, > new ProjectMergeRuleWithLongerName(), > EnumerableRules.ENUMERABLE_PROJECT_RULE, > EnumerableRules.ENUMERABLE_VALUES_RULE); > Program program = Programs.of(ruleSet); > RelTraitSet toTraits = > relNode.getCluster().traitSet() > .replace(0, EnumerableConvention.INSTANCE); > RelNode output = program.run(planner, relNode, toTraits, > ImmutableList.<RelOptMaterialization>of(), > ImmutableList.<RelOptLattice>of()); > // check for output > } > /** > * ProjectMergeRule inheritor which has > * class name greater than ProjectReduceExpressionsRule class name > (String.compareTo()). > * > * It is needed for RuleQueue.popMatch() method > * to apply this rule before ProjectReduceExpressionsRule. > */ > private static class ProjectMergeRuleWithLongerName extends > ProjectMergeRule { > public ProjectMergeRuleWithLongerName() { > super(true, RelFactories.LOGICAL_BUILDER); > } > } > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)