github-actions[bot] commented on code in PR #63899:
URL: https://github.com/apache/doris/pull/63899#discussion_r3711802591
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/PartitionIncrementMaintainer.java:
##########
@@ -803,6 +826,51 @@ public Plan getOriginalPlan() {
return originalPlan;
}
+ private Expression shuttleExpressionWithLineage(Expression expression,
Plan plan) {
+ return shuttleExpressionWithLineage(ImmutableList.of(expression),
plan).get(0);
+ }
+
+ private List<? extends Expression> shuttleExpressionWithLineage(List<?
extends Expression> expressions,
+ Plan plan) {
+ if (expressions.isEmpty()) {
+ return ImmutableList.of();
+ }
+ ExpressionLineageReplacer.ExpressionReplaceContext replaceContext =
+ new
ExpressionLineageReplacer.ExpressionReplaceContext(expressions);
+ for (NamedExpression namedExpression :
getLineageExpressionIndex(plan)) {
+ if
(!replaceContext.getUsedExprIdSet().contains(namedExpression.getExprId())) {
+ continue;
+ }
+
namedExpression.accept(ExpressionLineageReplacer.NamedExpressionCollector.INSTANCE,
replaceContext);
+ }
+ return replaceContext.getReplacedExpressions();
+ }
+
+ private List<? extends Expression>
shuttleAndNormalizeExpressionWithLineage(
+ Collection<? extends Expression> expressions, Plan plan) {
+ if (expressions.isEmpty()) {
+ return ImmutableList.of();
+ }
+ List<? extends Expression> shuttledExpressions =
+
shuttleExpressionWithLineage(ImmutableList.copyOf(expressions), plan);
+ List<Expression> normalizedExpressions = new
ArrayList<>(shuttledExpressions.size());
+ for (Expression expression : shuttledExpressions) {
+
normalizedExpressions.add(EXPRESSION_NORMALIZATION.rewrite(expression,
expressionRewriteContext));
+ }
+ return normalizedExpressions;
+ }
+
+ private List<NamedExpression> getLineageExpressionIndex(Plan plan) {
+ List<NamedExpression> lineageExpressionIndex =
planLineageExpressionIndexes.get(plan);
+ if (lineageExpressionIndex == null) {
+ List<NamedExpression> collectedIndex = new ArrayList<>();
+ plan.accept(LineageExpressionCollector.INSTANCE,
collectedIndex);
+ lineageExpressionIndex = collectedIndex;
+ planLineageExpressionIndexes.put(plan, lineageExpressionIndex);
Review Comment:
[P1] Avoid caching every nested join's whole lineage subtree
This context is not keyed only by `originalPlan`: every nonempty
`calEqualSet` calls `shuttleExpressionWithLineage(..., join)`, so a left-deep
projected join chain inserts each nested `LogicalJoin` identity here. For a
reduced tree with W aliases per level, the cached lists retain roughly WN +
W(N-1) + ... + W references (and perform the same quadratic collection work)
until the check ends; `UNION ALL` keeps all child contexts until merging. That
can trade the CPU problem this PR targets for FE heap/GC pressure on
derived-table/CTE join shapes. The new performance SQL joins scans directly, so
it does not exercise overlapping NamedExpression-heavy join subtrees. Please
avoid retaining a full subtree list per nested join (for example, use one root
index with subtree ranges or a non-retaining walk for nested keys) and add a
deterministic depth/entry-count test. This is distinct from the earlier
cache-scope thread: the current context-local map already receives multiple
overlapping jo
in subtrees.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
