[
https://issues.apache.org/jira/browse/SPARK-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14269038#comment-14269038
]
Gerard Maas commented on SPARK-4940:
------------------------------------
I forgot to mention that in the previous example, the ideal resource assignment
would be:
|| Mesos CPU || Mesos Mem || Spark tasks || Streaming receivers ||
| 3 | 4GB | 2 | 1 |
| 3 | 4GB | 2 | 1 |
| 3 | 4GB | 2 | 1 |
| 3 | 4GB | 2 | 1 |
| 3 | 4GB | 2 | 1 |
| 3 | 4GB | 2 | 1 |
Total resources: 18 CPUs, 24GB
This is an evenly distributed number CPUs and receivers over physical nodes to
maximize the spread of network receivers over nodes.
> Support more evenly distributing cores for Mesos mode
> -----------------------------------------------------
>
> Key: SPARK-4940
> URL: https://issues.apache.org/jira/browse/SPARK-4940
> Project: Spark
> Issue Type: Improvement
> Components: Mesos
> Reporter: Timothy Chen
> Attachments: mesos-config-difference-3nodes-vs-2nodes.png
>
>
> Currently in Coarse grain mode the spark scheduler simply takes all the
> resources it can on each node, but can cause uneven distribution based on
> resources available on each slave.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
