allisonwang-db commented on a change in pull request #32303:
URL: https://github.com/apache/spark/pull/32303#discussion_r647032535
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/DecorrelateInnerQuery.scala
##########
@@ -244,7 +244,17 @@ object DecorrelateInnerQuery extends PredicateHelper {
case _ => Join(child, domain, Inner, None, JoinHint.NONE)
}
} else {
- throw
QueryExecutionErrors.cannotRewriteDomainJoinWithConditionsError(conditions, d)
+ // Leave it unchanged when the domain join does not belong to the
current outer
+ // plan, i.e we cannot construct the domain attribute mapping from
the join conditions.
+ // This can happen when the outer plan itself has another
DomainJoin. E.g:
+ // LateralJoin lateral-subquery#263 [c1#262 && (c1#271 <=> c1#262)],
Inner
+ // : +- Project [c1#271 AS c1#265, c1#271]
+ // : +- DomainJoin [c1#271]
+ // : +- OneRowRelation
+ // +- Project [(c1#270 + 1) AS c1#262, c1#270]
+ // +- DomainJoin [c1#270]
+ // +- OneRowRelation
+ d
Review comment:
It will be rewritten during the optimization for the outer lateral
joins. You can think of each DomainJoin has a "scope" defined, i.e which outer
plan it belongs to, and here we only rewrite a domain join if it belongs to the
current outer plan.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
