[
https://issues.apache.org/jira/browse/YARN-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13678096#comment-13678096
]
Karthik Kambatla commented on YARN-648:
---------------------------------------
bq. What do you mean by latency? A policy that is compute-intensive can still
slow down things for everyone.
Agree, I should probably leave out the latency and just leave it as configure
appropriate policy.
However, I believe latency is also one of the reasons. Consider a configuration
where root.a and root.b are to parent queues, configured to use fairshare and
SP (a sophisticated scheduler with higher latency than fairshare) respectively.
root itself has fairshare. Now, the apps under root.a don't suffer from the
overhead of SP. Further, in the future, scheduling within a queue can be done
asynchronously to achieve even higher throughput and the time to schedule an
app depends on the cumulative latencies of parent queue schedulers? If I
understand it right, Omega (Google) was built to handle such latencies in the
presence of sophisticated schedulers.
> FS: Add documentation for pluggable policy
> ------------------------------------------
>
> Key: YARN-648
> URL: https://issues.apache.org/jira/browse/YARN-648
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 2.0.4-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Labels: documentaion
> Attachments: yarn-648-1.patch
>
>
> YARN-469 and YARN-482 make the scheduling policy in FS pluggable. Need to add
> documentation on how to use this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
