[
https://issues.apache.org/jira/browse/SPARK-17637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15515247#comment-15515247
]
Zhan Zhang commented on SPARK-17637:
------------------------------------
cc [~rxin]
A quick prototype shows that for a tested pipeline, the job can save around
45% regarding the reserved cpu and memory when the dynamic allocation is
enabled.
> Packed scheduling for Spark tasks across executors
> --------------------------------------------------
>
> Key: SPARK-17637
> URL: https://issues.apache.org/jira/browse/SPARK-17637
> Project: Spark
> Issue Type: Improvement
> Components: Scheduler
> Reporter: Zhan Zhang
> Priority: Minor
>
> Currently Spark scheduler implements round robin scheduling for tasks to
> executors. Which is great as it distributes the load evenly across the
> cluster, but this leads to significant resource waste in some cases,
> especially when dynamic allocation is enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
