uros-b commented on code in PR #59399:
URL: https://github.com/apache/spark/pull/59399#discussion_r4257593202
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/resolver/SubqueryScope.scala:
##########
@@ -74,8 +74,7 @@ import
org.apache.spark.sql.catalyst.plans.logical.{Aggregate, Filter, LogicalPl
* which indicates how many subquery levels we need to go up to find the
attribute reference.
* We need the outerScopeLevel map when resolving the subquery expressions,
where we collect
* the outer references and wrap the ones cannot be resolved in the current
subquery scope or
- * its parent scopes with [[OuterScopeReference]]. More details can be found
in
- * `SubqueryExpressionResolver.processOuterReferences`.
+ * its parent scopes with [[OuterScopeReference]].
Review Comment:
This still says unresolved outer refs are wrapped in OuterScopeReference.
processOuterReferences is already just SubExprUtils.getOuterReferences. You
cleaned the sibling comment and left this one.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/resolver/SubqueryExpressionResolver.scala:
##########
@@ -201,58 +201,8 @@ class SubqueryExpressionResolver(expressionResolver:
ExpressionResolver, resolve
}
/**
- * Return `outerAttrs` for the resolved subquery plan.
- * If `spark.sql.optimizer.supportNestedCorrelatedSubqueries.enabled` is set
to false, simply
- * collect the outer references within the subquery plan.
- *
- * Otherwise,
- * 1. Get the outer references within the resolved subquery plan and also
get the outer
- * attributes from any nested subqueries.
- * 2. For each outer reference, using the `outerReferenceScopeLevel` map
from current subquery
- * scope to check if it has only one level of correlation or beyond one
level of correlation.
- * If it is beyond one level of correlation, wrap it with
[[OuterScopeReference]] and add it to
- * the parent subquery scope's `outerReferenceScopeLevel` map with level -
1. E.g: For the
- * following query:
- *
- * {{{
- * SELECT *
- * FROM t1
- * WHERE t1.a = (
- * SELECT MAX(t2.a)
- * FROM t2
- * WHERE t2.a = (
- * SELECT MAX(t3.a)
- * FROM t3
- * WHERE t3.b > t2.b AND t3.c > t1.c
- * ) AND t2.b > t1.b
- * }}}
- *
- * After resolving:
- *
- * {{{
- * Aggregate (MAX(t3.a))
- * Filter (t3.b > OuterReference(t2.b) AND t3.c > OuterReference(t1.c))
- * Scan t3
- * }}}
- *
- * The innermost subquery has two outer references:
`t2.b(outerScopeReferenceLevel = 1)` and
- * `t1.c(outerScopeReferenceLevel = 2)`. The innermost subquery expression
is resolved as
- * `scalar-subquery(outerAttrs = [t2.b, OuterScopeReference(t1.c)])` and
- * `(t1.c.exprId -> outerScopeReferenceLevel = 1)` is added to the
`outerReferenceScopeLevel`
- * map of the parent subquery scope.
- * After resolving the parent subquery plan:
- *
- * {{{
- * Aggregate (MAX(t2.a))
- * Filter (t2.a = scalar-subquery(outerAttrs = [t2.b,
OuterScopeReference(t1.c)]) AND
- * t2.b > OuterReference(t1.b))
- * Scan t2
- * }}}
- *
- * The collected outer references are `t1.b` and the
`OuterScopeReference(t1.c)` from the
- * innermost subquery. Both of them have `outerScopeReferenceLevel = 1`,
they are all resolved
- * as one level outer references. The subquery expression is then resolved as
- * `scalar-subquery(outerAttrs = [t1.b, t1.c])`.
+ * Return `outerAttrs` for the resolved subquery plan by collecting the
outer references within
+ * the subquery plan. Only one level of correlation is supported.
*/
private def processOuterReferences(
resolvedPlan: LogicalPlan,
Review Comment:
Is parentSubqueryScope used? Can we remove it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
