[jira] [Commented] (CALCITE-3376) VolcanoPlanner CannotPlanException: best rel is null even though there is an option with non-infinite cost
[
https://issues.apache.org/jira/browse/CALCITE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962850#comment-16962850
]
Ruben Q L commented on CALCITE-3376:
Unit test merged via
https://github.com/apache/calcite/commit/464d520148f7106fe580eec9419373276865564e
> VolcanoPlanner CannotPlanException: best rel is null even though there is an
> option with non-infinite cost
> --
>
> Key: CALCITE-3376
> URL: https://issues.apache.org/jira/browse/CALCITE-3376
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
> Labels: pull-request-available
> Attachments: Diagram.png, Graphviz.png, logsTRACE.txt, stackTrace.txt
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The problem can be reproduced by adding this test to PlannerTest.java:
> {code:java}
> @Test public void testCannotPlanException() throws Exception {
> RelBuilder builder = RelBuilder.create(RelBuilderTest.config().build());
> RuleSet ruleSet =
> RuleSets.ofList(
> //EnumerableRules.ENUMERABLE_JOIN_RULE, // with this rule it
> works!
> JoinToCorrelateRule.INSTANCE,
> EnumerableRules.ENUMERABLE_CORRELATE_RULE,
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_UNION_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> builder
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .join(
> JoinRelType.INNER,
> builder.equals(
> builder.field(2, 0, "DEPTNO"),
> builder.field(2, 1, "EMPNO")));
> RelNode relNode = builder.build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits = relNode.getTraitSet()
> .replace(EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.of(), ImmutableList.of());
> String outputStr = toString(output);
> }
> {code}
> Running this test causes the following exception (full stack trace attached):
> {code:java}
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
> enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[]. All the inputs have relevant nodes, however
> the cost is still infinite.
> Root: rel#13:Subset#2.ENUMERABLE.[]
> {code}
> The last part of the message (_All the inputs have relevant nodes, however
> the cost is still infinite_) seems relevant, because we can see that
> {{rel#13}}'s best is {{null}}, and it should be {{rel#21}} (which has a
> non-infinite cost):
> {code:java}
> rel#13:Subset#2.ENUMERABLE.[], best=null, importance=1.0
> rel#14:AbstractConverter.ENUMERABLE.[](input=RelSubset#12,
> convention=ENUMERABLE, sort=[]), rowcount=117.6, cumulative cost={inf}
> rel#21:EnumerableCorrelate.ENUMERABLE.[](left=RelSubset#19,
> right=RelSubset#20, correlation=$cor0, joinType=inner, requiredColumns={7}),
> rowcount=1.0, cumulative cost={1770.60001 rows, 2466.0 cpu, 0.0 io}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3376) VolcanoPlanner CannotPlanException: best rel is null even though there is an option with non-infinite cost
[
https://issues.apache.org/jira/browse/CALCITE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962811#comment-16962811
]
Ruben Q L commented on CALCITE-3376:
As expected after the merge of CALCITE-2018, I have re-tested with master and
the current issue is solved. Thanks [~xndai] for pointing me in the right
direction.
I'll open a PR anyway, just to add a unit test to make sure we can detect this
kind of issue if we ever have a regression in the future.
> VolcanoPlanner CannotPlanException: best rel is null even though there is an
> option with non-infinite cost
> --
>
> Key: CALCITE-3376
> URL: https://issues.apache.org/jira/browse/CALCITE-3376
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
> Attachments: Diagram.png, Graphviz.png, logsTRACE.txt, stackTrace.txt
>
>
> The problem can be reproduced by adding this test to PlannerTest.java:
> {code:java}
> @Test public void testCannotPlanException() throws Exception {
> RelBuilder builder = RelBuilder.create(RelBuilderTest.config().build());
> RuleSet ruleSet =
> RuleSets.ofList(
> //EnumerableRules.ENUMERABLE_JOIN_RULE, // with this rule it
> works!
> JoinToCorrelateRule.INSTANCE,
> EnumerableRules.ENUMERABLE_CORRELATE_RULE,
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_UNION_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> builder
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .join(
> JoinRelType.INNER,
> builder.equals(
> builder.field(2, 0, "DEPTNO"),
> builder.field(2, 1, "EMPNO")));
> RelNode relNode = builder.build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits = relNode.getTraitSet()
> .replace(EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.of(), ImmutableList.of());
> String outputStr = toString(output);
> }
> {code}
> Running this test causes the following exception (full stack trace attached):
> {code:java}
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
> enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[]. All the inputs have relevant nodes, however
> the cost is still infinite.
> Root: rel#13:Subset#2.ENUMERABLE.[]
> {code}
> The last part of the message (_All the inputs have relevant nodes, however
> the cost is still infinite_) seems relevant, because we can see that
> {{rel#13}}'s best is {{null}}, and it should be {{rel#21}} (which has a
> non-infinite cost):
> {code:java}
> rel#13:Subset#2.ENUMERABLE.[], best=null, importance=1.0
> rel#14:AbstractConverter.ENUMERABLE.[](input=RelSubset#12,
> convention=ENUMERABLE, sort=[]), rowcount=117.6, cumulative cost={inf}
> rel#21:EnumerableCorrelate.ENUMERABLE.[](left=RelSubset#19,
> right=RelSubset#20, correlation=$cor0, joinType=inner, requiredColumns={7}),
> rowcount=1.0, cumulative cost={1770.60001 rows, 2466.0 cpu, 0.0 io}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3376) VolcanoPlanner CannotPlanException: best rel is null even though there is an option with non-infinite cost
[
https://issues.apache.org/jira/browse/CALCITE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954710#comment-16954710
]
Xiening Dai commented on CALCITE-3376:
--
Great to know. Thanks for checking.
> VolcanoPlanner CannotPlanException: best rel is null even though there is an
> option with non-infinite cost
> --
>
> Key: CALCITE-3376
> URL: https://issues.apache.org/jira/browse/CALCITE-3376
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: Diagram.png, Graphviz.png, logsTRACE.txt, stackTrace.txt
>
>
> The problem can be reproduced by adding this test to PlannerTest.java:
> {code:java}
> @Test public void testCannotPlanException() throws Exception {
> RelBuilder builder = RelBuilder.create(RelBuilderTest.config().build());
> RuleSet ruleSet =
> RuleSets.ofList(
> //EnumerableRules.ENUMERABLE_JOIN_RULE, // with this rule it
> works!
> JoinToCorrelateRule.INSTANCE,
> EnumerableRules.ENUMERABLE_CORRELATE_RULE,
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_UNION_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> builder
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .join(
> JoinRelType.INNER,
> builder.equals(
> builder.field(2, 0, "DEPTNO"),
> builder.field(2, 1, "EMPNO")));
> RelNode relNode = builder.build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits = relNode.getTraitSet()
> .replace(EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.of(), ImmutableList.of());
> String outputStr = toString(output);
> }
> {code}
> Running this test causes the following exception (full stack trace attached):
> {code:java}
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
> enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[]. All the inputs have relevant nodes, however
> the cost is still infinite.
> Root: rel#13:Subset#2.ENUMERABLE.[]
> {code}
> The last part of the message (_All the inputs have relevant nodes, however
> the cost is still infinite_) seems relevant, because we can see that
> {{rel#13}}'s best is {{null}}, and it should be {{rel#21}} (which has a
> non-infinite cost):
> {code:java}
> rel#13:Subset#2.ENUMERABLE.[], best=null, importance=1.0
> rel#14:AbstractConverter.ENUMERABLE.[](input=RelSubset#12,
> convention=ENUMERABLE, sort=[]), rowcount=117.6, cumulative cost={inf}
> rel#21:EnumerableCorrelate.ENUMERABLE.[](left=RelSubset#19,
> right=RelSubset#20, correlation=$cor0, joinType=inner, requiredColumns={7}),
> rowcount=1.0, cumulative cost={1770.60001 rows, 2466.0 cpu, 0.0 io}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3376) VolcanoPlanner CannotPlanException: best rel is null even though there is an option with non-infinite cost
[
https://issues.apache.org/jira/browse/CALCITE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954462#comment-16954462
]
Ruben Q L commented on CALCITE-3376:
[~xndai], sorry for the late reply, I have checked the unit test that triggers
the current issue with your branch https://github.com/xndai/calcite/tree/2018
and the test runs successfully. So it seems you are right, this issue will be
solved by CALCITE-2018. Thank you very much for the feedback.
I will wait until CALCITE-2018 gets merged into master, and then re-try, if the
current issue is solved, I will close the current ticket.
> VolcanoPlanner CannotPlanException: best rel is null even though there is an
> option with non-infinite cost
> --
>
> Key: CALCITE-3376
> URL: https://issues.apache.org/jira/browse/CALCITE-3376
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: Diagram.png, Graphviz.png, logsTRACE.txt, stackTrace.txt
>
>
> The problem can be reproduced by adding this test to PlannerTest.java:
> {code:java}
> @Test public void testCannotPlanException() throws Exception {
> RelBuilder builder = RelBuilder.create(RelBuilderTest.config().build());
> RuleSet ruleSet =
> RuleSets.ofList(
> //EnumerableRules.ENUMERABLE_JOIN_RULE, // with this rule it
> works!
> JoinToCorrelateRule.INSTANCE,
> EnumerableRules.ENUMERABLE_CORRELATE_RULE,
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_UNION_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> builder
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .join(
> JoinRelType.INNER,
> builder.equals(
> builder.field(2, 0, "DEPTNO"),
> builder.field(2, 1, "EMPNO")));
> RelNode relNode = builder.build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits = relNode.getTraitSet()
> .replace(EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.of(), ImmutableList.of());
> String outputStr = toString(output);
> }
> {code}
> Running this test causes the following exception (full stack trace attached):
> {code:java}
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
> enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[]. All the inputs have relevant nodes, however
> the cost is still infinite.
> Root: rel#13:Subset#2.ENUMERABLE.[]
> {code}
> The last part of the message (_All the inputs have relevant nodes, however
> the cost is still infinite_) seems relevant, because we can see that
> {{rel#13}}'s best is {{null}}, and it should be {{rel#21}} (which has a
> non-infinite cost):
> {code:java}
> rel#13:Subset#2.ENUMERABLE.[], best=null, importance=1.0
> rel#14:AbstractConverter.ENUMERABLE.[](input=RelSubset#12,
> convention=ENUMERABLE, sort=[]), rowcount=117.6, cumulative cost={inf}
> rel#21:EnumerableCorrelate.ENUMERABLE.[](left=RelSubset#19,
> right=RelSubset#20, correlation=$cor0, joinType=inner, requiredColumns={7}),
> rowcount=1.0, cumulative cost={1770.60001 rows, 2466.0 cpu, 0.0 io}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3376) VolcanoPlanner CannotPlanException: best rel is null even though there is an option with non-infinite cost
[
https://issues.apache.org/jira/browse/CALCITE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16949062#comment-16949062
]
Wang Yanlin commented on CALCITE-3376:
--
This seems to be fixed after https://issues.apache.org/jira/browse/CALCITE-3330
merged.
> VolcanoPlanner CannotPlanException: best rel is null even though there is an
> option with non-infinite cost
> --
>
> Key: CALCITE-3376
> URL: https://issues.apache.org/jira/browse/CALCITE-3376
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: Diagram.png, Graphviz.png, logsTRACE.txt, stackTrace.txt
>
>
> The problem can be reproduced by adding this test to PlannerTest.java:
> {code:java}
> @Test public void testCannotPlanException() throws Exception {
> RelBuilder builder = RelBuilder.create(RelBuilderTest.config().build());
> RuleSet ruleSet =
> RuleSets.ofList(
> //EnumerableRules.ENUMERABLE_JOIN_RULE, // with this rule it
> works!
> JoinToCorrelateRule.INSTANCE,
> EnumerableRules.ENUMERABLE_CORRELATE_RULE,
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_UNION_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> builder
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .join(
> JoinRelType.INNER,
> builder.equals(
> builder.field(2, 0, "DEPTNO"),
> builder.field(2, 1, "EMPNO")));
> RelNode relNode = builder.build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits = relNode.getTraitSet()
> .replace(EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.of(), ImmutableList.of());
> String outputStr = toString(output);
> }
> {code}
> Running this test causes the following exception (full stack trace attached):
> {code:java}
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
> enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[]. All the inputs have relevant nodes, however
> the cost is still infinite.
> Root: rel#13:Subset#2.ENUMERABLE.[]
> {code}
> The last part of the message (_All the inputs have relevant nodes, however
> the cost is still infinite_) seems relevant, because we can see that
> {{rel#13}}'s best is {{null}}, and it should be {{rel#21}} (which has a
> non-infinite cost):
> {code:java}
> rel#13:Subset#2.ENUMERABLE.[], best=null, importance=1.0
> rel#14:AbstractConverter.ENUMERABLE.[](input=RelSubset#12,
> convention=ENUMERABLE, sort=[]), rowcount=117.6, cumulative cost={inf}
> rel#21:EnumerableCorrelate.ENUMERABLE.[](left=RelSubset#19,
> right=RelSubset#20, correlation=$cor0, joinType=inner, requiredColumns={7}),
> rowcount=1.0, cumulative cost={1770.60001 rows, 2466.0 cpu, 0.0 io}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3376) VolcanoPlanner CannotPlanException: best rel is null even though there is an option with non-infinite cost
[
https://issues.apache.org/jira/browse/CALCITE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940699#comment-16940699
]
Ruben Q L commented on CALCITE-3376:
Thanks for the info [~xndai]. I'm traveling right now, but I'll try it as soon
as possible.
> VolcanoPlanner CannotPlanException: best rel is null even though there is an
> option with non-infinite cost
> --
>
> Key: CALCITE-3376
> URL: https://issues.apache.org/jira/browse/CALCITE-3376
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: Diagram.png, Graphviz.png, logsTRACE.txt, stackTrace.txt
>
>
> The problem can be reproduced by adding this test to PlannerTest.java:
> {code:java}
> @Test public void testCannotPlanException() throws Exception {
> RelBuilder builder = RelBuilder.create(RelBuilderTest.config().build());
> RuleSet ruleSet =
> RuleSets.ofList(
> //EnumerableRules.ENUMERABLE_JOIN_RULE, // with this rule it
> works!
> JoinToCorrelateRule.INSTANCE,
> EnumerableRules.ENUMERABLE_CORRELATE_RULE,
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_UNION_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> builder
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .join(
> JoinRelType.INNER,
> builder.equals(
> builder.field(2, 0, "DEPTNO"),
> builder.field(2, 1, "EMPNO")));
> RelNode relNode = builder.build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits = relNode.getTraitSet()
> .replace(EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.of(), ImmutableList.of());
> String outputStr = toString(output);
> }
> {code}
> Running this test causes the following exception (full stack trace attached):
> {code:java}
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
> enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[]. All the inputs have relevant nodes, however
> the cost is still infinite.
> Root: rel#13:Subset#2.ENUMERABLE.[]
> {code}
> The last part of the message (_All the inputs have relevant nodes, however
> the cost is still infinite_) seems relevant, because we can see that
> {{rel#13}}'s best is {{null}}, and it should be {{rel#21}} (which has a
> non-infinite cost):
> {code:java}
> rel#13:Subset#2.ENUMERABLE.[], best=null, importance=1.0
> rel#14:AbstractConverter.ENUMERABLE.[](input=RelSubset#12,
> convention=ENUMERABLE, sort=[]), rowcount=117.6, cumulative cost={inf}
> rel#21:EnumerableCorrelate.ENUMERABLE.[](left=RelSubset#19,
> right=RelSubset#20, correlation=$cor0, joinType=inner, requiredColumns={7}),
> rowcount=1.0, cumulative cost={1770.60001 rows, 2466.0 cpu, 0.0 io}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3376) VolcanoPlanner CannotPlanException: best rel is null even though there is an option with non-infinite cost
[
https://issues.apache.org/jira/browse/CALCITE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940621#comment-16940621
]
Xiening Dai commented on CALCITE-3376:
--
Thanks [~rubenql] for root causing this issue. This is exactly one of the
scenarios which CALCITE-2018 tries to fix. In current fix proposal, it clears
the parent RelNode's cost from the cache before propogating cost improvement to
the parent. Please try it out. It should fix this issue.
> VolcanoPlanner CannotPlanException: best rel is null even though there is an
> option with non-infinite cost
> --
>
> Key: CALCITE-3376
> URL: https://issues.apache.org/jira/browse/CALCITE-3376
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: Diagram.png, Graphviz.png, logsTRACE.txt, stackTrace.txt
>
>
> The problem can be reproduced by adding this test to PlannerTest.java:
> {code:java}
> @Test public void testCannotPlanException() throws Exception {
> RelBuilder builder = RelBuilder.create(RelBuilderTest.config().build());
> RuleSet ruleSet =
> RuleSets.ofList(
> //EnumerableRules.ENUMERABLE_JOIN_RULE, // with this rule it
> works!
> JoinToCorrelateRule.INSTANCE,
> EnumerableRules.ENUMERABLE_CORRELATE_RULE,
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_UNION_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> builder
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .join(
> JoinRelType.INNER,
> builder.equals(
> builder.field(2, 0, "DEPTNO"),
> builder.field(2, 1, "EMPNO")));
> RelNode relNode = builder.build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits = relNode.getTraitSet()
> .replace(EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.of(), ImmutableList.of());
> String outputStr = toString(output);
> }
> {code}
> Running this test causes the following exception (full stack trace attached):
> {code:java}
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
> enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[]. All the inputs have relevant nodes, however
> the cost is still infinite.
> Root: rel#13:Subset#2.ENUMERABLE.[]
> {code}
> The last part of the message (_All the inputs have relevant nodes, however
> the cost is still infinite_) seems relevant, because we can see that
> {{rel#13}}'s best is {{null}}, and it should be {{rel#21}} (which has a
> non-infinite cost):
> {code:java}
> rel#13:Subset#2.ENUMERABLE.[], best=null, importance=1.0
> rel#14:AbstractConverter.ENUMERABLE.[](input=RelSubset#12,
> convention=ENUMERABLE, sort=[]), rowcount=117.6, cumulative cost={inf}
> rel#21:EnumerableCorrelate.ENUMERABLE.[](left=RelSubset#19,
> right=RelSubset#20, correlation=$cor0, joinType=inner, requiredColumns={7}),
> rowcount=1.0, cumulative cost={1770.60001 rows, 2466.0 cpu, 0.0 io}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3376) VolcanoPlanner CannotPlanException: best rel is null even though there is an option with non-infinite cost
[
https://issues.apache.org/jira/browse/CALCITE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16939845#comment-16939845
]
Haisheng Yuan commented on CALCITE-3376:
[~xndai] FYI
> VolcanoPlanner CannotPlanException: best rel is null even though there is an
> option with non-infinite cost
> --
>
> Key: CALCITE-3376
> URL: https://issues.apache.org/jira/browse/CALCITE-3376
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: Diagram.png, Graphviz.png, logsTRACE.txt, stackTrace.txt
>
>
> The problem can be reproduced by adding this test to PlannerTest.java:
> {code:java}
> @Test public void testCannotPlanException() throws Exception {
> RelBuilder builder = RelBuilder.create(RelBuilderTest.config().build());
> RuleSet ruleSet =
> RuleSets.ofList(
> //EnumerableRules.ENUMERABLE_JOIN_RULE, // with this rule it
> works!
> JoinToCorrelateRule.INSTANCE,
> EnumerableRules.ENUMERABLE_CORRELATE_RULE,
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_UNION_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> builder
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .join(
> JoinRelType.INNER,
> builder.equals(
> builder.field(2, 0, "DEPTNO"),
> builder.field(2, 1, "EMPNO")));
> RelNode relNode = builder.build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits = relNode.getTraitSet()
> .replace(EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.of(), ImmutableList.of());
> String outputStr = toString(output);
> }
> {code}
> Running this test causes the following exception (full stack trace attached):
> {code:java}
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
> enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[]. All the inputs have relevant nodes, however
> the cost is still infinite.
> Root: rel#13:Subset#2.ENUMERABLE.[]
> {code}
> The last part of the message (_All the inputs have relevant nodes, however
> the cost is still infinite_) seems relevant, because we can see that
> {{rel#13}}'s best is {{null}}, and it should be {{rel#21}} (which has a
> non-infinite cost):
> {code:java}
> rel#13:Subset#2.ENUMERABLE.[], best=null, importance=1.0
> rel#14:AbstractConverter.ENUMERABLE.[](input=RelSubset#12,
> convention=ENUMERABLE, sort=[]), rowcount=117.6, cumulative cost={inf}
> rel#21:EnumerableCorrelate.ENUMERABLE.[](left=RelSubset#19,
> right=RelSubset#20, correlation=$cor0, joinType=inner, requiredColumns={7}),
> rowcount=1.0, cumulative cost={1770.60001 rows, 2466.0 cpu, 0.0 io}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3376) VolcanoPlanner CannotPlanException: best rel is null even though there is an option with non-infinite cost
[
https://issues.apache.org/jira/browse/CALCITE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16939600#comment-16939600
]
Ruben Q L commented on CALCITE-3376:
Thanks for the info, [~danny0405]. I haven't checked CALCITE-2018, I'll try to
test it when I have some time.
> VolcanoPlanner CannotPlanException: best rel is null even though there is an
> option with non-infinite cost
> --
>
> Key: CALCITE-3376
> URL: https://issues.apache.org/jira/browse/CALCITE-3376
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: Diagram.png, Graphviz.png, logsTRACE.txt, stackTrace.txt
>
>
> The problem can be reproduced by adding this test to PlannerTest.java:
> {code:java}
> @Test public void testCannotPlanException() throws Exception {
> RelBuilder builder = RelBuilder.create(RelBuilderTest.config().build());
> RuleSet ruleSet =
> RuleSets.ofList(
> //EnumerableRules.ENUMERABLE_JOIN_RULE, // with this rule it
> works!
> JoinToCorrelateRule.INSTANCE,
> EnumerableRules.ENUMERABLE_CORRELATE_RULE,
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_UNION_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> builder
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .join(
> JoinRelType.INNER,
> builder.equals(
> builder.field(2, 0, "DEPTNO"),
> builder.field(2, 1, "EMPNO")));
> RelNode relNode = builder.build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits = relNode.getTraitSet()
> .replace(EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.of(), ImmutableList.of());
> String outputStr = toString(output);
> }
> {code}
> Running this test causes the following exception (full stack trace attached):
> {code:java}
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
> enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[]. All the inputs have relevant nodes, however
> the cost is still infinite.
> Root: rel#13:Subset#2.ENUMERABLE.[]
> {code}
> The last part of the message (_All the inputs have relevant nodes, however
> the cost is still infinite_) seems relevant, because we can see that
> {{rel#13}}'s best is {{null}}, and it should be {{rel#21}} (which has a
> non-infinite cost):
> {code:java}
> rel#13:Subset#2.ENUMERABLE.[], best=null, importance=1.0
> rel#14:AbstractConverter.ENUMERABLE.[](input=RelSubset#12,
> convention=ENUMERABLE, sort=[]), rowcount=117.6, cumulative cost={inf}
> rel#21:EnumerableCorrelate.ENUMERABLE.[](left=RelSubset#19,
> right=RelSubset#20, correlation=$cor0, joinType=inner, requiredColumns={7}),
> rowcount=1.0, cumulative cost={1770.60001 rows, 2466.0 cpu, 0.0 io}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3376) VolcanoPlanner CannotPlanException: best rel is null even though there is an option with non-infinite cost
[
https://issues.apache.org/jira/browse/CALCITE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16939547#comment-16939547
]
Danny Chen commented on CALCITE-3376:
-
[~rubenql] Did you try CALCITE-2018 ? It fix some cases of metadata cache, can
you revert change 2166 to see it that causes the problem, I might take some
time to see 2018 tomorrow.
> VolcanoPlanner CannotPlanException: best rel is null even though there is an
> option with non-infinite cost
> --
>
> Key: CALCITE-3376
> URL: https://issues.apache.org/jira/browse/CALCITE-3376
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: Diagram.png, Graphviz.png, logsTRACE.txt, stackTrace.txt
>
>
> The problem can be reproduced by adding this test to PlannerTest.java:
> {code:java}
> @Test public void testCannotPlanException() throws Exception {
> RelBuilder builder = RelBuilder.create(RelBuilderTest.config().build());
> RuleSet ruleSet =
> RuleSets.ofList(
> //EnumerableRules.ENUMERABLE_JOIN_RULE, // with this rule it
> works!
> JoinToCorrelateRule.INSTANCE,
> EnumerableRules.ENUMERABLE_CORRELATE_RULE,
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_UNION_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> builder
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .join(
> JoinRelType.INNER,
> builder.equals(
> builder.field(2, 0, "DEPTNO"),
> builder.field(2, 1, "EMPNO")));
> RelNode relNode = builder.build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits = relNode.getTraitSet()
> .replace(EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.of(), ImmutableList.of());
> String outputStr = toString(output);
> }
> {code}
> Running this test causes the following exception (full stack trace attached):
> {code:java}
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
> enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[]. All the inputs have relevant nodes, however
> the cost is still infinite.
> Root: rel#13:Subset#2.ENUMERABLE.[]
> {code}
> The last part of the message (_All the inputs have relevant nodes, however
> the cost is still infinite_) seems relevant, because we can see that
> {{rel#13}}'s best is {{null}}, and it should be {{rel#21}} (which has a
> non-infinite cost):
> {code:java}
> rel#13:Subset#2.ENUMERABLE.[], best=null, importance=1.0
> rel#14:AbstractConverter.ENUMERABLE.[](input=RelSubset#12,
> convention=ENUMERABLE, sort=[]), rowcount=117.6, cumulative cost={inf}
> rel#21:EnumerableCorrelate.ENUMERABLE.[](left=RelSubset#19,
> right=RelSubset#20, correlation=$cor0, joinType=inner, requiredColumns={7}),
> rowcount=1.0, cumulative cost={1770.60001 rows, 2466.0 cpu, 0.0 io}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3376) VolcanoPlanner CannotPlanException: best rel is null even though there is an option with non-infinite cost
[
https://issues.apache.org/jira/browse/CALCITE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16939540#comment-16939540
]
Ruben Q L commented on CALCITE-3376:
No sure if this might be related to CALCITE-2166
> VolcanoPlanner CannotPlanException: best rel is null even though there is an
> option with non-infinite cost
> --
>
> Key: CALCITE-3376
> URL: https://issues.apache.org/jira/browse/CALCITE-3376
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: Diagram.png, Graphviz.png, logsTRACE.txt, stackTrace.txt
>
>
> The problem can be reproduced by adding this test to PlannerTest.java:
> {code:java}
> @Test public void testCannotPlanException() throws Exception {
> RelBuilder builder = RelBuilder.create(RelBuilderTest.config().build());
> RuleSet ruleSet =
> RuleSets.ofList(
> //EnumerableRules.ENUMERABLE_JOIN_RULE, // with this rule it
> works!
> JoinToCorrelateRule.INSTANCE,
> EnumerableRules.ENUMERABLE_CORRELATE_RULE,
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_UNION_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> builder
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .join(
> JoinRelType.INNER,
> builder.equals(
> builder.field(2, 0, "DEPTNO"),
> builder.field(2, 1, "EMPNO")));
> RelNode relNode = builder.build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits = relNode.getTraitSet()
> .replace(EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.of(), ImmutableList.of());
> String outputStr = toString(output);
> }
> {code}
> Running this test causes the following exception (full stack trace attached):
> {code:java}
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
> enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[]. All the inputs have relevant nodes, however
> the cost is still infinite.
> Root: rel#13:Subset#2.ENUMERABLE.[]
> {code}
> The last part of the message (_All the inputs have relevant nodes, however
> the cost is still infinite_) seems relevant, because we can see that
> {{rel#13}}'s best is {{null}}, and it should be {{rel#21}} (which has a
> non-infinite cost):
> {code:java}
> rel#13:Subset#2.ENUMERABLE.[], best=null, importance=1.0
> rel#14:AbstractConverter.ENUMERABLE.[](input=RelSubset#12,
> convention=ENUMERABLE, sort=[]), rowcount=117.6, cumulative cost={inf}
> rel#21:EnumerableCorrelate.ENUMERABLE.[](left=RelSubset#19,
> right=RelSubset#20, correlation=$cor0, joinType=inner, requiredColumns={7}),
> rowcount=1.0, cumulative cost={1770.60001 rows, 2466.0 cpu, 0.0 io}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3376) VolcanoPlanner CannotPlanException: best rel is null even though there is an option with non-infinite cost
[
https://issues.apache.org/jira/browse/CALCITE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16939364#comment-16939364
]
Ruben Q L commented on CALCITE-3376:
The explanation on the previous comment seems to be in the right direction,
because this simple change in {{VolcanoPlanner#getCost}} to force the rel's
cost computation using {{computeSelfCost}} (instead of {{RelMetadataQuery}}
info) avoids the exception:
{code}
public RelOptCost getCost(RelNode rel, RelMetadataQuery mq) {
...
//RelOptCost cost = mq.getNonCumulativeCost(rel);
RelOptCost cost = rel.computeSelfCost(rel.getCluster().getPlanner(), mq);
...
{code}
I am not saying that this is a valid fix for the problem, but it is just
another element to confirm that the issue is located in that part of the
process.
> VolcanoPlanner CannotPlanException: best rel is null even though there is an
> option with non-infinite cost
> --
>
> Key: CALCITE-3376
> URL: https://issues.apache.org/jira/browse/CALCITE-3376
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: Diagram.png, Graphviz.png, logsTRACE.txt, stackTrace.txt
>
>
> The problem can be reproduced by adding this test to PlannerTest.java:
> {code:java}
> @Test public void testCannotPlanException() throws Exception {
> RelBuilder builder = RelBuilder.create(RelBuilderTest.config().build());
> RuleSet ruleSet =
> RuleSets.ofList(
> //EnumerableRules.ENUMERABLE_JOIN_RULE, // with this rule it
> works!
> JoinToCorrelateRule.INSTANCE,
> EnumerableRules.ENUMERABLE_CORRELATE_RULE,
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_UNION_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> builder
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .join(
> JoinRelType.INNER,
> builder.equals(
> builder.field(2, 0, "DEPTNO"),
> builder.field(2, 1, "EMPNO")));
> RelNode relNode = builder.build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits = relNode.getTraitSet()
> .replace(EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.of(), ImmutableList.of());
> String outputStr = toString(output);
> }
> {code}
> Running this test causes the following exception (full stack trace attached):
> {code:java}
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
> enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[]. All the inputs have relevant nodes, however
> the cost is still infinite.
> Root: rel#13:Subset#2.ENUMERABLE.[]
> {code}
> The last part of the message (_All the inputs have relevant nodes, however
> the cost is still infinite_) seems relevant, because we can see that
> {{rel#13}}'s best is {{null}}, and it should be {{rel#21}} (which has a
> non-infinite cost):
> {code:java}
> rel#13:Subset#2.ENUMERABLE.[], best=null, importance=1.0
> rel#14:AbstractConverter.ENUMERABLE.[](input=RelSubset#12,
> convention=ENUMERABLE, sort=[]), rowcount=117.6, cumulative cost={inf}
> rel#21:EnumerableCorrelate.ENUMERABLE.[](left=RelSubset#19,
> right=RelSubset#20, correlation=$cor0, joinType=inner, requiredColumns={7}),
> rowcount=1.0, cumulative cost={1770.60001 rows, 2466.0 cpu, 0.0 io}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3376) VolcanoPlanner CannotPlanException: best rel is null even though there is an option with non-infinite cost
[
https://issues.apache.org/jira/browse/CALCITE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16939342#comment-16939342
]
Ruben Q L commented on CALCITE-3376:
After some debugging, _I think I know_ what is going on. Taken the example from
the description, during the optimization phase we have this situation:
!Diagram.png!
Note that the Union is both left input of the Correlate and input of the Filter.
At some point, when the relevant rules have been applied, the planner starts
getting costs that are better than the initial ones (infinite), see
{{RelSubset#propagateCostImprovements}}. As we can see in the attached file
logsTRACE.txt, firstly it happens with the scan (rel#23):
{code}
Subset cost changed: subset [rel#23:Subset#0.ENUMERABLE.[0]] cost was {inf} now
{14.0 rows, 15.0 cpu, 0.0 io}
{code}
Then, this cost change is propagated to its parent, i.e. the Union (rel#19):
{code}
Subset cost changed: subset [rel#19:Subset#1.ENUMERABLE.[]] cost was {inf} now
{56.0 rows, 58.0 cpu, 0.0 io}
{code}
Then, it continues the propagation to the Union's parents, which are two: the
Correlate and the Filter (both still with infinite cost). First it goes to the
Correlate (rel#13), but when re-computing its cost, there is no change, it is
still infinite because even though its left child (Union) has a valid cost, its
right child (Filter) has infinite cost (yet). Next, the next Union's parent is
processed: the Filter (rel#20), and its cost is improved:
{code}
Subset cost changed: subset [rel#20:Subset#3.ENUMERABLE.[]] cost was {inf} now
{60.2 rows, 86.0 cpu, 0.0 io}
{code}
Finally, the propagation continues to the Filter's parent: the Correlate, so
again its cost is (re)evaluated. Unlike the previous calculation, at this point
the Correlate will have a valid cost, since both children (Union and Filter)
have valid costs. However, when computing its cost, it is still infinite, that
leads to the CannotPlanException. Why it is still infinite? The debugger shows
that this second computation does not actually reach
{{Correlate#computeSelfCost}}, in fact, it would seem that the first cost
computation (infinite) was "cached" by the RelMetadataQuery, so it (wrongly)
returns its cached value (infinite) when computing the Correlate cost the
second time. Unfortunately, this seems a bad decision, because an actual
computation of {{Correlate#computeSelfCost}} would have returned a valid cost
value this time.
> VolcanoPlanner CannotPlanException: best rel is null even though there is an
> option with non-infinite cost
> --
>
> Key: CALCITE-3376
> URL: https://issues.apache.org/jira/browse/CALCITE-3376
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: Diagram.png, Graphviz.png, logsTRACE.txt, stackTrace.txt
>
>
> The problem can be reproduced by adding this test to PlannerTest.java:
> {code:java}
> @Test public void testCannotPlanException() throws Exception {
> RelBuilder builder = RelBuilder.create(RelBuilderTest.config().build());
> RuleSet ruleSet =
> RuleSets.ofList(
> //EnumerableRules.ENUMERABLE_JOIN_RULE, // with this rule it
> works!
> JoinToCorrelateRule.INSTANCE,
> EnumerableRules.ENUMERABLE_CORRELATE_RULE,
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_UNION_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> builder
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .join(
> JoinRelType.INNER,
> builder.equals(
> builder.field(2, 0, "DEPTNO"),
> builder.field(2, 1, "EMPNO")));
> RelNode relNode = builder.build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits = relNode.getTraitSet()
> .replace(EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.of(), ImmutableList.of());
> String outputStr = toString(output);
> }
> {code}
> Running this test causes the following exception (full stack trace attached):
> {code:java}
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
> enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[]. All the inputs have relevant nodes, however
> the cost is still infinite.
> Root: rel#13:Subset#2.ENUMERABLE.[]
> {code}
> The
[jira] [Commented] (CALCITE-3376) VolcanoPlanner CannotPlanException: best rel is null even though there is an option with non-infinite cost
[
https://issues.apache.org/jira/browse/CALCITE-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16939194#comment-16939194
]
Ruben Q L commented on CALCITE-3376:
What I have found so far:
The problem seems to happen with the following circumstances:
- There is a Join (of any type) implemented as a Correlate; note that if in the
test we active the {{ENUMERABLE_JOIN_RULE}}, the final join would be
implemented as a HashJoin instead of as a Correlate, and the tests would pass.
- The left and right inputs of the Correlate must be equal (not just to equal
scans, but something more elaborate). In the test we have two UNION ALL, but if
we changed one (only one) of the unions to all=false, the test would pass. A
more complex plan that follows this pattern and that triggers the problem could
be:
{code}
builder
.scan("EMP")
.filter(
builder.equals(
builder.field("DEPTNO"),
builder.literal(1)))
.scan("EMP")
.filter(
builder.equals(
builder.field("DEPTNO"),
builder.literal(2)))
.union(true)
.scan("EMP")
.filter(
builder.equals(
builder.field("DEPTNO"),
builder.literal(1)))
.scan("EMP")
.filter(
builder.equals(
builder.field("DEPTNO"),
builder.literal(2)))
.union(true)
.antiJoin(
builder.call(
SqlStdOperatorTable.GREATER_THAN,
builder.field(2, 1, "SAL"),
builder.field(2, 0, "SAL")));
{code}
> VolcanoPlanner CannotPlanException: best rel is null even though there is an
> option with non-infinite cost
> --
>
> Key: CALCITE-3376
> URL: https://issues.apache.org/jira/browse/CALCITE-3376
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Ruben Q L
>Priority: Major
> Attachments: stackTrace.txt
>
>
> The problem can be reproduced by adding this test to PlannerTest.java:
> {code:java}
> @Test public void testCannotPlanException() throws Exception {
> RelBuilder builder = RelBuilder.create(RelBuilderTest.config().build());
> RuleSet ruleSet =
> RuleSets.ofList(
> //EnumerableRules.ENUMERABLE_JOIN_RULE, // with this rule it
> works!
> JoinToCorrelateRule.INSTANCE,
> EnumerableRules.ENUMERABLE_CORRELATE_RULE,
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_UNION_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> builder
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .scan("EMP")
> .scan("EMP")
> .union(true)
> .join(
> JoinRelType.INNER,
> builder.equals(
> builder.field(2, 0, "DEPTNO"),
> builder.field(2, 1, "EMPNO")));
> RelNode relNode = builder.build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits = relNode.getTraitSet()
> .replace(EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.of(), ImmutableList.of());
> String outputStr = toString(output);
> }
> {code}
> Running this test causes the following exception (full stack trace attached):
> {code:java}
> org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not
> enough rules to produce a node with desired properties:
> convention=ENUMERABLE, sort=[]. All the inputs have relevant nodes, however
> the cost is still infinite.
> Root: rel#13:Subset#2.ENUMERABLE.[]
> {code}
> The last part of the message (_All the inputs have relevant nodes, however
> the cost is still infinite_) seems relevant, because we can see that
> {{rel#13}}'s best is {{null}}, and it should be {{rel#21}} (which has a
> non-infinite cost):
> {code:java}
> rel#13:Subset#2.ENUMERABLE.[], best=null, importance=1.0
> rel#14:AbstractConverter.ENUMERABLE.[](input=RelSubset#12,
> convention=ENUMERABLE, sort=[]), rowcount=117.6, cumulative cost={inf}
> rel#21:EnumerableCorrelate.ENUMERABLE.[](left=RelSubset#19,
> right=RelSubset#20, correlation=$cor0, joinType=inner, requiredColumns={7}),
> rowcount=1.0, cumulative cost={1770.60001 rows, 2466.0 cpu, 0.0 io}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
