Re: OOM and "spark.buffer.pageSize"

2016-03-28 Thread Steve Johnston
Yes I have. That’s the best source of information at the moment. Thanks. -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/OOM-and-spark-buffer-pageSize-tp16890p16892.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

Re: OOM and "spark.buffer.pageSize"

2016-03-28 Thread Ted Yu
I guess you have looked at MemoryManager#pageSizeBytes where the "spark.buffer.pageSize" config can override default page size. FYI On Mon, Mar 28, 2016 at 12:07 PM, Steve Johnston < sjohns...@algebraixdata.com> wrote: > I'm attempting to address an OOM issue. I saw referenced in >

OOM and "spark.buffer.pageSize"

2016-03-28 Thread Steve Johnston
I'm attempting to address an OOM issue. I saw referenced in java.lang.OutOfMemoryError: Unable to acquire bytes of memory the configuration setting