Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/20368#discussion_r163373026
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/joins/BroadcastJoinSuite.scala
---
@@ -126,6 +126,22 @@ class BroadcastJoinSuite extends QueryTest with
SQLTestUtils {
}
}
+ test("broadcast hint is retained in a cached plan") {
+ Seq(true, false).foreach { materialized =>
+ withSQLConf(SQLConf.AUTO_BROADCASTJOIN_THRESHOLD.key -> "-1") {
+ val df1 = spark.createDataFrame(Seq((1, "4"), (2,
"2"))).toDF("key", "value")
+ val df2 = spark.createDataFrame(Seq((1, "1"), (2,
"2"))).toDF("key", "value")
+ broadcast(df2).cache()
+ if (materialized) df2.collect()
--- End diff --
This PR https://github.com/apache/spark/pull/19864 accidentally fixes the
issue when the plan is not materialized. However, it does not resolve the issue
when the cached plan is materialized.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
