[
https://issues.apache.org/jira/browse/CALCITE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18065354#comment-18065354
]
Wenzhuang Zhu commented on CALCITE-6907:
----------------------------------------
Usage of large plan optimization:
{code:java}
HepPlanner planner = new HepPlanner();
// or use other constructor and set isLargePlanMode/enableFiredRulesCache = true
// HepPlanner planner = new HepPlanner(new HepProgramBuilder().build(), ...);
// planner.setEnableFiredRulesCache(true);
// planner.setLargePlanMode(true);
planner.setRoot(initPlanRoot);
planner.executeProgram(phase1Program);
planner.dumpRuleAttemptsInfo(); // optional
planner.clear(); // clear the rules and rule match caches, the graph is reused
// other logic ...
planner.executeProgram(phase2Program);
planner.clear();
// ...
RelNode optimized = planner.buildFinalPlan(); {code}
> Optimize latency for large plan
> -------------------------------
>
> Key: CALCITE-6907
> URL: https://issues.apache.org/jira/browse/CALCITE-6907
> Project: Calcite
> Issue Type: Improvement
> Reporter: Wenzhuang Zhu
> Assignee: Wenzhuang Zhu
> Priority: Major
> Labels: pull-request-available
>
> *HepPlanner:*
> HepPlanner cannot quickly optimize plan with 10K+ rel nodes.
> We can improve it by:
> 1.HepPlanner support graph reuse for different HepProgram.
> 2.Fast graph GC on discardedVertex's subtree. Don't do whole graph GC.
> 3.Use large plan friendly graph iterator.
>
> {*}RelDataTypeDigest{*}:
> For nested Type with 1K+ inner field, computeDigest for composite type is too
> heavy.
> RelDataTypeDigest is similar to RelDigest. It will cache hash code and
> prevent generate type string for composite type.
>
> *RelOnceVisitor:*
> Deep nested CTE can be expanded to more than 100 millions nodes if we don't
> do dedup during visiting Rel tree.
> BTW, explain has the same problem, needs a better string explain format for
> duplicate RelNode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
