fair share scheduler does not utilize all slots if the task trackers are
configured heterogeneously
---------------------------------------------------------------------------------------------------
Key: HADOOP-4943
URL: https://issues.apache.org/jira/browse/HADOOP-4943
Project: Hadoop Core
Issue Type: Bug
Components: contrib/fair-share
Affects Versions: 0.19.0
Reporter: Zheng Shao
Assignee: Zheng Shao
There is some code in the fairshare scheduler that tries to make the load
across the whole cluster the same.
That piece of code will break if the task trackers are configured differently.
Basically, we will stop assigning more tasks to tasks trackers that have tasks
above the cluster average, but we may still want to do that because other task
trackers may have less slots.
We should change the code to maintain a cluster-wide slot usage percentage
(instead of absolute number of slot usage) to make sure the load is evenly
distributed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
