I'm attempting to address an OOM issue. I saw referenced in java.lang.OutOfMemoryError: Unable to acquire bytes of memory <http://apache-spark-developers-list.1001551.n3.nabble.com/java-lang-OutOfMemoryError-Unable-to-acquire-bytes-of-memory-td16773.html> the configuration setting "spark.buffer.pageSize" which was used in conjunction with "spark.sql.shuffle.partitions" to solve the OOM problem Nezih was having.
What is "spark.buffer.pageSize"? How can it be used? I can find it in the code but there doesn't seem to be any other documentation. Thanks, Steve -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/OOM-and-spark-buffer-pageSize-tp16890.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
