I guess you have looked at MemoryManager#pageSizeBytes where
the "spark.buffer.pageSize" config can override default page size.
FYI
On Mon, Mar 28, 2016 at 12:07 PM, Steve Johnston <
sjohns...@algebraixdata.com> wrote:
> I'm attempting to address an OOM issue. I saw referenced in
> java.lang.OutOfMemoryError: Unable to acquire bytes of memory
> <
> http://apache-spark-developers-list.1001551.n3.nabble.com/java-lang-OutOfMemoryError-Unable-to-acquire-bytes-of-memory-td16773.html
> >
> the configuration setting "spark.buffer.pageSize" which was used in
> conjunction with "spark.sql.shuffle.partitions" to solve the OOM problem
> Nezih was having.
>
> What is "spark.buffer.pageSize"? How can it be used? I can find it in the
> code but there doesn't seem to be any other documentation.
>
> Thanks,
> Steve
>
>
>
> --
> View this message in context:
> http://apache-spark-developers-list.1001551.n3.nabble.com/OOM-and-spark-buffer-pageSize-tp16890.html
> Sent from the Apache Spark Developers List mailing list archive at
> Nabble.com.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
> For additional commands, e-mail: dev-h...@spark.apache.org
>
>
