allisonwang-db commented on a change in pull request #32303: URL: https://github.com/apache/spark/pull/32303#discussion_r647032535
########## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/DecorrelateInnerQuery.scala ########## @@ -244,7 +244,17 @@ object DecorrelateInnerQuery extends PredicateHelper { case _ => Join(child, domain, Inner, None, JoinHint.NONE) } } else { - throw QueryExecutionErrors.cannotRewriteDomainJoinWithConditionsError(conditions, d) + // Leave it unchanged when the domain join does not belong to the current outer + // plan, i.e we cannot construct the domain attribute mapping from the join conditions. + // This can happen when the outer plan itself has another DomainJoin. E.g: + // LateralJoin lateral-subquery#263 [c1#262 && (c1#271 <=> c1#262)], Inner + // : +- Project [c1#271 AS c1#265, c1#271] + // : +- DomainJoin [c1#271] + // : +- OneRowRelation + // +- Project [(c1#270 + 1) AS c1#262, c1#270] + // +- DomainJoin [c1#270] + // +- OneRowRelation + d Review comment: It will be rewritten during the optimization for the outer lateral joins. You can think of each DomainJoin has a "scope" defined, i.e which outer plan it belongs to, and here we only rewrite a domain join if it belongs to the current outer plan. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org