Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19050#discussion_r137142974
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala
---
@@ -49,6 +49,30 @@ object RewritePredicateSubquery extends
Rule[LogicalPlan] with PredicateHelper {
}
}
+ def dedupJoin(plan: LogicalPlan): LogicalPlan = {
+ plan transform {
+ case j @ Join(left, right, joinType, joinCond) =>
--- End diff --
Be safe, this only makes sense for `LeftAnti ` and `LeftSemi `
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
