Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20407#discussion_r167392027 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -156,6 +156,15 @@ object SQLConf { .booleanConf .createWithDefault(true) + val ALLOW_NESTEDJOIN_FALLBACK = buildConf("spark.sql.join.broadcastJoinFallback.enabled") + .internal() + .doc("When true (default), if the other options are not available, fallback to try and use " + + "BroadcastNestedLoopJoin as join strategy. This can cause OOM which can be a problem " + + "in some scenarios, eg. when running the thriftserver. Turn to false to disable it: an " + + "AnalysisException will be thrown.") --- End diff -- why? May you please explain which would be the right direction then?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org