[jira] [Commented] (CALCITE-3916) Support cascades style top-down driven rule apply
[
https://issues.apache.org/jira/browse/CALCITE-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176821#comment-17176821
]
Ruben Q L commented on CALCITE-3916:
Thanks [~hyuan]! That solves the {{ClassCastException}} issue. Sorry, my bad, I
misunderstood the {{passThrough}} contract, I thought that, apart from
collation, I had to consider convention too. But you are right, ignoring the
convention fixed the problem.
> Support cascades style top-down driven rule apply
> -
>
> Key: CALCITE-3916
> URL: https://issues.apache.org/jira/browse/CALCITE-3916
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Haisheng Yuan
>Assignee: Jinpeng Wu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.24.0
>
> Time Spent: 15.5h
> Remaining Estimate: 0h
>
> Apply rules by leaf RelSet -> root RelSet order. For every RelNode in a
> RelSet, rule is matched and applied sequentially. No RuleQueue and
> DeferringRuleCall is needed anymore. This will make space pruning and rule
> mutual exclusivity check possible.
> Rule that use AbstractConverter as operand is an exception, to keep backward
> compatibility, this kind of rule still needs top-down apply.
> This should be done after CALCITE-3896.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3916) Support cascades style top-down driven rule apply
[ https://issues.apache.org/jira/browse/CALCITE-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176726#comment-17176726 ] Haisheng Yuan commented on CALCITE-3916: Thanks for your detailed explanation., [~rubenql]! We need to investigate the root cause of the regression for your pattern, I can imagine it might hard to create a repro test case, but if you can, that would be great. Usually the operator(IndexScan45) generated from "passThrough" should better have the same convention with original operator (IndexScan38), ignore the required "ENUMERABLE" convention, have you tried that? > Support cascades style top-down driven rule apply > - > > Key: CALCITE-3916 > URL: https://issues.apache.org/jira/browse/CALCITE-3916 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Assignee: Jinpeng Wu >Priority: Major > Labels: pull-request-available > Fix For: 1.24.0 > > Time Spent: 15.5h > Remaining Estimate: 0h > > Apply rules by leaf RelSet -> root RelSet order. For every RelNode in a > RelSet, rule is matched and applied sequentially. No RuleQueue and > DeferringRuleCall is needed anymore. This will make space pruning and rule > mutual exclusivity check possible. > Rule that use AbstractConverter as operand is an exception, to keep backward > compatibility, this kind of rule still needs top-down apply. > This should be done after CALCITE-3896. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3916) Support cascades style top-down driven rule apply
[
https://issues.apache.org/jira/browse/CALCITE-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175352#comment-17175352
]
Ruben Q L commented on CALCITE-3916:
Thanks for checking [~hyuan], grosso modo the situation is as follows:
I have an IndexScan operator which can perform an scan with filter and/or sort
within. With a certain rule, the first IndexScan is created with a filter
inside:
{{Filter + Scan => IndexScan[id 38, filter]}}
Then, the method "passThrough" is called on this IndexScan38. The required
traiset contains a collation, so a new IndexScan is created (using 38 as
"base"), ending up with IndexScan45, which contains the filter and sort
(collation):
{{createdSortedIndexScan(IndexScan38, collation) => IndexScan[id 45, filter,
collation]}}
Since "passThrough" requires also ENUMERABLE Convention, and my IndexScan
operator belongs to a different one, this method returns a
MyConventionToEnumerableConverter(with IndexScan45 inside).
The {{ClassCastException}} is thrown when this IndexScan45 is tried to be cast
as RelSubset inside {{TopDownRuleDriver#getOptimizeInputTask}}.
> Support cascades style top-down driven rule apply
> -
>
> Key: CALCITE-3916
> URL: https://issues.apache.org/jira/browse/CALCITE-3916
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Haisheng Yuan
>Assignee: Jinpeng Wu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.24.0
>
> Time Spent: 15.5h
> Remaining Estimate: 0h
>
> Apply rules by leaf RelSet -> root RelSet order. For every RelNode in a
> RelSet, rule is matched and applied sequentially. No RuleQueue and
> DeferringRuleCall is needed anymore. This will make space pruning and rule
> mutual exclusivity check possible.
> Rule that use AbstractConverter as operand is an exception, to keep backward
> compatibility, this kind of rule still needs top-down apply.
> This should be done after CALCITE-3896.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3916) Support cascades style top-down driven rule apply
[ https://issues.apache.org/jira/browse/CALCITE-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175077#comment-17175077 ] Haisheng Yuan commented on CALCITE-3916: thanks for reporting, [~rubenql]. how is the scan operator with the specific rel id generated? > Support cascades style top-down driven rule apply > - > > Key: CALCITE-3916 > URL: https://issues.apache.org/jira/browse/CALCITE-3916 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Assignee: Jinpeng Wu >Priority: Major > Labels: pull-request-available > Fix For: 1.24.0 > > Time Spent: 15.5h > Remaining Estimate: 0h > > Apply rules by leaf RelSet -> root RelSet order. For every RelNode in a > RelSet, rule is matched and applied sequentially. No RuleQueue and > DeferringRuleCall is needed anymore. This will make space pruning and rule > mutual exclusivity check possible. > Rule that use AbstractConverter as operand is an exception, to keep backward > compatibility, this kind of rule still needs top-down apply. > This should be done after CALCITE-3896. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3916) Support cascades style top-down driven rule apply
[
https://issues.apache.org/jira/browse/CALCITE-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174374#comment-17174374
]
Ruben Q L commented on CALCITE-3916:
[~FatLittle], [~hyuan], FYI, I have a prototype with TopDown activated, and
recently I updated it and this change causes a {{ClassCastException}} here:
https://github.com/apache/calcite/blob/aadb605decd6bb6a853e23fac4b0f479b2397e06/core/src/main/java/org/apache/calcite/plan/volcano/TopDownRuleDriver.java#L548
with message:
{code}
...
Caused by: java.lang.ClassCastException: MyTableScanOperator cannot be cast to
com.onwbp.org.apache.calcite.plan.volcano.RelSubset
at
com.onwbp.org.apache.calcite.plan.volcano.TopDownRuleDriver.getOptimizeInputTask(TopDownRuleDriver.java:548)
...
{code}
Does it make sense, or maybe I am doing something wrong in my prototype? (I do
not know if I can reproduce the situation with a unit test in Calcite)
> Support cascades style top-down driven rule apply
> -
>
> Key: CALCITE-3916
> URL: https://issues.apache.org/jira/browse/CALCITE-3916
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Haisheng Yuan
>Assignee: Jinpeng Wu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.24.0
>
> Time Spent: 15.5h
> Remaining Estimate: 0h
>
> Apply rules by leaf RelSet -> root RelSet order. For every RelNode in a
> RelSet, rule is matched and applied sequentially. No RuleQueue and
> DeferringRuleCall is needed anymore. This will make space pruning and rule
> mutual exclusivity check possible.
> Rule that use AbstractConverter as operand is an exception, to keep backward
> compatibility, this kind of rule still needs top-down apply.
> This should be done after CALCITE-3896.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3916) Support cascades style top-down driven rule apply
[ https://issues.apache.org/jira/browse/CALCITE-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157108#comment-17157108 ] Haisheng Yuan commented on CALCITE-3916: Yes, doing the final round review > Support cascades style top-down driven rule apply > - > > Key: CALCITE-3916 > URL: https://issues.apache.org/jira/browse/CALCITE-3916 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Assignee: Jinpeng Wu >Priority: Major > Labels: pull-request-available > Fix For: 1.24.0 > > Time Spent: 12h 40m > Remaining Estimate: 0h > > Apply rules by leaf RelSet -> root RelSet order. For every RelNode in a > RelSet, rule is matched and applied sequentially. No RuleQueue and > DeferringRuleCall is needed anymore. This will make space pruning and rule > mutual exclusivity check possible. > Rule that use AbstractConverter as operand is an exception, to keep backward > compatibility, this kind of rule still needs top-down apply. > This should be done after CALCITE-3896. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3916) Support cascades style top-down driven rule apply
[ https://issues.apache.org/jira/browse/CALCITE-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17157106#comment-17157106 ] Chunwei Lei commented on CALCITE-3916: -- Hi, [~FatLittle] and [~hyuan], do you think the PR is good enough to be merged into 1.24? > Support cascades style top-down driven rule apply > - > > Key: CALCITE-3916 > URL: https://issues.apache.org/jira/browse/CALCITE-3916 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Assignee: Jinpeng Wu >Priority: Major > Labels: pull-request-available > Fix For: 1.24.0 > > Time Spent: 12.5h > Remaining Estimate: 0h > > Apply rules by leaf RelSet -> root RelSet order. For every RelNode in a > RelSet, rule is matched and applied sequentially. No RuleQueue and > DeferringRuleCall is needed anymore. This will make space pruning and rule > mutual exclusivity check possible. > Rule that use AbstractConverter as operand is an exception, to keep backward > compatibility, this kind of rule still needs top-down apply. > This should be done after CALCITE-3896. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3916) Support cascades style top-down driven rule apply
[ https://issues.apache.org/jira/browse/CALCITE-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100596#comment-17100596 ] Jinpeng Wu commented on CALCITE-3916: - PR: https://github.com/apache/calcite/pull/1950 There might be two ways to accomplish this. The first one is designing another Planner while the second is modifying the VolcanoPlanner directly and make sure it won't break the current logic. The pros and cons are discussed: https://lists.apache.org/thread.html/r38ea71968c069f465921e7197488329c15413b46831c90ad4d48f87e%40%3Cdev.calcite.apache.org%3E My code is now generally on the first track. Currently it should not be difficult to switch to the second one. However, I am still trying some aggressive optimizations. So I am not going to take the second way until many people insist. Thanks > Support cascades style top-down driven rule apply > - > > Key: CALCITE-3916 > URL: https://issues.apache.org/jira/browse/CALCITE-3916 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Assignee: Jinpeng Wu >Priority: Major > > Apply rules by leaf RelSet -> root RelSet order. For every RelNode in a > RelSet, rule is matched and applied sequentially. No RuleQueue and > DeferringRuleCall is needed anymore. This will make space pruning and rule > mutual exclusivity check possible. > Rule that use AbstractConverter as operand is an exception, to keep backward > compatibility, this kind of rule still needs top-down apply. > This should be done after CALCITE-3896. -- This message was sent by Atlassian Jira (v8.3.4#803005)
