[GitHub] [spark] wangyum commented on a change in pull request #31857: [SPARK-28220][SQL] Push down the foldable predicate to both sides of Join

[GitBox]

wangyum commented on a change in pull request #31857:
URL: https://github.com/apache/spark/pull/31857#discussion_r595688798



##########
File path: 
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala
##########
@@ -1384,4 +1384,14 @@ class FilterPushdownSuite extends PlanTest {
       condition = Some("x.a".attr === "z.a".attr)).analyze
     comparePlans(optimized, correctAnswer)
   }
+
+  test("SPARK-28220: Push down the foldable predicate to both sides of Join") {
+    val x = testRelation.subquery('x)
+    val y = testRelation.subquery('y)
+    val originalQuery = x.join(y, condition = Some(false))
+
+    val optimized = Optimize.execute(originalQuery.analyze)
+    val correctAnswer = x.where(false).join(y.where(false), condition = 
None).analyze
+    comparePlans(optimized, correctAnswer.analyze)

Review comment:
       We use `Inner` join to do test because it need `PruneFilters` if use 
`LeftOuter` join:
   ```scala
   val x = testRelation.subquery('x)
   val y = testRelation.subquery('y)
   val originalQuery = x.join(y, LeftOuter, condition = Some(false))
   
   val optimized = Optimize.execute(originalQuery.analyze)
   val correctAnswer = x.join(y.where(false), LeftOuter, condition = 
None).analyze
   comparePlans(optimized, correctAnswer.analyze)
   ```
   Error:
   ```
   == FAIL: Plans do not match ===
    'Join LeftOuter, false                            'Join LeftOuter, false
    :- LocalRelation <empty>, [a#0, b#0, c#0]         :- LocalRelation <empty>, 
[a#0, b#0, c#0]
    +- Filter false                                   +- Filter false
   !   +- Filter false                                   +- LocalRelation 
<empty>, [a#0, b#0, c#0]
   !      +- LocalRelation <empty>, [a#0, b#0, c#0]   
   ```
   
   There are lots of test case fail if add `PruneFilters`:
   
![image](https://user-images.githubusercontent.com/5399861/111411368-68f6cd80-8715-11eb-9964-bf4129936201.png)
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org