Dhiraj created SPARK-23427:
------------------------------
Summary: spark.sql.autoBroadcastJoinThreshold causing OOM in the
driver
Key: SPARK-23427
URL: https://issues.apache.org/jira/browse/SPARK-23427
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.0.0
Environment: SPARK 2.0 version
Reporter: Dhiraj
We are facing issue around value of spark.sql.autoBroadcastJoinThreshold.
With spark.sql.autoBroadcastJoinThreshold -1 ( disable) we seeing driver memory
used flat.
With any other values 10MB, 5MB, 2 MB, 1MB, 10K, 1K we see driver memory used
goes up with rate depending upon the size of the autoBroadcastThreshold and
getting OOM exception. The problem is memory used by autoBroadcast is not being
free up in the driver.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
