[
https://issues.apache.org/jira/browse/SPARK-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihui updated SPARK-2193:
--------------------------
Description:
Now, the last executor(s) maybe not get it’s preferred task(s), although these
tasks have build in pendingTasksForHosts map. Because executers pick up tasks
sequential, their preferred task(s) maybe picked up by other executors.
This appearance can be eliminated by sorting tasks partial ordering. Executor
pick up task by host’s order of task’s preferredLocation, that mean, executor
firstly pick up all tasks which task.preferredLocations.1 = executor.hostName,
then secondly…
> Improve tasks‘ preferred locality by sorting tasks partial ordering
> -------------------------------------------------------------------
>
> Key: SPARK-2193
> URL: https://issues.apache.org/jira/browse/SPARK-2193
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 1.1.0
> Reporter: Zhihui
>
> Now, the last executor(s) maybe not get it’s preferred task(s), although
> these tasks have build in pendingTasksForHosts map. Because executers pick up
> tasks sequential, their preferred task(s) maybe picked up by other executors.
> This appearance can be eliminated by sorting tasks partial ordering. Executor
> pick up task by host’s order of task’s preferredLocation, that mean, executor
> firstly pick up all tasks which task.preferredLocations.1 =
> executor.hostName, then secondly…
--
This message was sent by Atlassian JIRA
(v6.2#6252)
