Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20407#discussion_r167392509 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -156,6 +156,15 @@ object SQLConf { .booleanConf .createWithDefault(true) + val ALLOW_NESTEDJOIN_FALLBACK = buildConf("spark.sql.join.broadcastJoinFallback.enabled") + .internal() + .doc("When true (default), if the other options are not available, fallback to try and use " + + "BroadcastNestedLoopJoin as join strategy. This can cause OOM which can be a problem " + + "in some scenarios, eg. when running the thriftserver. Turn to false to disable it: an " + + "AnalysisException will be thrown.") --- End diff -- OOM could be triggered by various cases. `BroadcastNestedLoopJoin` is being widely used in many cases. If this is the case in your scenario, you can do it in your fork. Also, this PR just helps a very specific case. I do not think it resolves the general issue.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org