[GitHub] [spark] wangyum commented on a change in pull request #31857: [SPARK-28220][SQL] Push down the foldable predicate to both sides of Join

[GitBox]

wangyum commented on a change in pull request #31857:
URL: https://github.com/apache/spark/pull/31857#discussion_r597072815



##########
File path: 
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/PropagateEmptyRelation.scala
##########
@@ -30,6 +31,7 @@ import org.apache.spark.sql.catalyst.rules._
  *    - Join with one or two empty children (including Intersect/Except).
  * 2. Unary-node Logical Plans
  *    - Project/Filter/Sample/Join/Limit/Repartition with all empty children.
+ *    - Join with false condition.

Review comment:
       Mainly used to avoid unnecessary pushdown:
   ```sql
   SELECT * FROM t1 LEFT JOIN t2 ON true
   ```
   Plan:
   ```
   === Applying Rule org.apache.spark.sql.catalyst.optimizer.PushDownPredicates 
===
    Join LeftOuter, true                        Join LeftOuter, true
    :- Relation default.t1[a#6L,b#7L] parquet   :- Relation 
default.t1[a#6L,b#7L] parquet
   !+- Relation default.t2[x#8L,y#9L] parquet   +- Filter true
   !                                               +- Relation 
default.t2[x#8L,y#9L] parquet
   
   === Applying Rule org.apache.spark.sql.catalyst.optimizer.PruneFilters ===
    Join LeftOuter, true                           Join LeftOuter, true
    :- Relation default.t1[a#6L,b#7L] parquet      :- Relation 
default.t1[a#6L,b#7L] parquet
   !+- Filter true                                 +- Relation 
default.t2[x#8L,y#9L] parquet
   !   +- Relation default.t2[x#8L,y#9L] parquet   
              
   00:57:41.206 ERROR org.apache.spark.sql.catalyst.rules.PlanChangeLogger: 
   === Result of Batch Cleanup filters that cannot be pushed down ===
    Join LeftOuter, true                           Join LeftOuter, true
    :- Relation default.t1[a#6L,b#7L] parquet      :- Relation 
default.t1[a#6L,b#7L] parquet
   !+- Filter true                                 +- Relation 
default.t2[x#8L,y#9L] parquet
   !   +- Relation default.t2[x#8L,y#9L] parquet 
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org