[
https://issues.apache.org/jira/browse/SPARK-18248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vincent gromakowski updated SPARK-18248:
----------------------------------------
Description:
Spark mesos framework doesn't reserve the size of offheap zone. It is only
taking the heap + 400 MB overhead into consideration for resources allocation.
Example: spark.executor.memory=3g spark.memory.offheap.size=1g ==> mesos report
3.4g allocated for the executor
The workaround is to set spark.mesos.executor.overhead to be at least
spark.memory.offheap.size
was:
Spark mesos framework is only taking the heap + 400 MB overhead into
consideration for resources allocation.
Example: spark.executor.memory=3g spark.memory.offheap.size=1g ==> mesos report
3.4g allocated for the executor
The workaround is to set spark.mesos.executor.overhead to be at least
spark.memory.offheap.size
> spark on mesos memory sizing with offheap
> -----------------------------------------
>
> Key: SPARK-18248
> URL: https://issues.apache.org/jira/browse/SPARK-18248
> Project: Spark
> Issue Type: Bug
> Components: Mesos
> Environment: Spark on Mesos
> Reporter: vincent gromakowski
>
> Spark mesos framework doesn't reserve the size of offheap zone. It is only
> taking the heap + 400 MB overhead into consideration for resources allocation.
> Example: spark.executor.memory=3g spark.memory.offheap.size=1g ==> mesos
> report 3.4g allocated for the executor
> The workaround is to set spark.mesos.executor.overhead to be at least
> spark.memory.offheap.size
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
