pratham76 commented on code in PR #58656:
URL: https://github.com/apache/spark/pull/58656#discussion_r4040725362
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala:
##########
@@ -439,7 +454,8 @@ object RewritePredicateSubquery extends Rule[LogicalPlan]
with PredicateHelper {
ExistenceJoin(exists), Some(finalJoinCond), joinHint)
introducedAttrs += exists
Not(exists)
- case InSubquery(values, ListQuery(sub, _, _, _, conditions, subHint))
=>
+ case sq @ InSubquery(values, ListQuery(sub, _, _, _, conditions,
subHint))
+ if canRewrite(sq) =>
Review Comment:
Not intended — thanks for spotting it. `transformDownWithPruning` recurses
into the children even
when the rule returns the node unchanged (it compares with `fastEquals` and
still calls
`mapChildren`), so returning the node as-is was not enough to stop the
descent. Since
`SubqueryExpression.children` is `outerAttrs ++ joinCond`, a sub-query
nested in a declined
leftover's own join condition was rewritten, grafting an existence join for
an `exists` reference
that lands in a join condition which `PlanSubqueries` may drop — I could see
it in the plan for your
doubly nested shape:
```
list#128 [c1#131 && c1#131 && ((col1#132 = c1#131) OR exists#140)]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
