Wangda Tan created YARN-5906:
--------------------------------
Summary: Update AppSchedulingInfo to use SchedulingPlacementSet
Key: YARN-5906
URL: https://issues.apache.org/jira/browse/YARN-5906
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Wangda Tan
Assignee: Wangda Tan
Currently AppSchedulingInfo simply stores resource request and scheduler make
decision according to stored resource request. For example, CS/FS use slightly
different approach to get pending resource request and make delay scheduling
decision.
There're several benefits of moving pending resource request data structure to
SchedulingPlacementSet
1) Delay scheduling logic should be agnostic to scheduler, for example CS
supports count-based delay and FS supports both of count-based and time-based
delay. Ideally scheduler should be able to choose which delay scheduling policy
to use.
2) In addition to 1., YARN-4902 has proposal to support pluggable delay
scheduling behavior in addition to locality-based (host->rack->offswitch).
Which requires more flexibility.
3) To make YARN-4902 becomes real, instead of directly adding the new resource
request API to client, we can make scheduler to use it internally to make sure
it is well defined. And AppSchedulingInfo/SchedulingPlacementSet will be the
perfect place to isolate which ResourceRequest implementation to use.
4) Different scheduling requirement needs different behavior of checking
ResourceRequest table.
This JIRA is the 1st patch of several refactorings. Which moves all
ResourceRequest data structure and logics to SchedulingPlacementSet. We need
follow changes to make it better structured
- Make delay scheduling to be a plugin of SchedulingPlacementSet
- After YARN-4902 get committed, change SchedulingPlacementSet to use YARN-4902
internally.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
