[
https://issues.apache.org/jira/browse/YARN-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15765321#comment-15765321
]
Wangda Tan commented on YARN-5906:
----------------------------------
Thanks reviews from [~sunilg].
bq. LocalitySchedulingPlacementSet implements few common placementset apis
Were you suggesting to move some implementations to an abstract base class to
share between other implementations, or were you suggesting to move some
private methods to the SchedulingPlacementSet interface?
bq. 3. AppSchedulingInfo#getResourceRequests return empty map ...
The major reason is to avoid null pointer check, and the emptyMap is a static
variable so it should not have any performance issue. Do you is it necessary to
expose "null" map to external caller?
bq. LocalitySchedulingPlacementSet has writelock and readlock ...
I can elaborate more: In my mind, SchedulingPlacementSet could be accessed
externally, for example, if we have any resource update, or container from
other application completed (for example inter-app antiaffinity). We can notify
the SchedulingPlacementSet. I think using the same AppSchedulingInfo may not
good enough. it should not risky because it doesn't invoke methods from any
other classes.
bq. 5. LocalitySchedulingPlacementSet supports singleNode from ..
Not sure if I understand the question, different SchedulingPlacementSet could
have different ordering of preferred node. Could you elaborate?
Attaching ver.4 patch which addressed #1 with some minor updates.
> Update AppSchedulingInfo to use SchedulingPlacementSet
> ------------------------------------------------------
>
> Key: YARN-5906
> URL: https://issues.apache.org/jira/browse/YARN-5906
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Attachments: YARN-5906.1.patch, YARN-5906.2.patch, YARN-5906.3.patch,
> YARN-5906.4.patch
>
>
> Currently AppSchedulingInfo simply stores resource request and scheduler make
> decision according to stored resource request. For example, CS/FS use
> slightly different approach to get pending resource request and make delay
> scheduling decision.
> There're several benefits of moving pending resource request data structure
> to SchedulingPlacementSet
> 1) Delay scheduling logic should be agnostic to scheduler, for example CS
> supports count-based delay and FS supports both of count-based and time-based
> delay. Ideally scheduler should be able to choose which delay scheduling
> policy to use.
> 2) In addition to 1., YARN-4902 has proposal to support pluggable delay
> scheduling behavior in addition to locality-based (host->rack->offswitch).
> Which requires more flexibility.
> 3) To make YARN-4902 becomes real, instead of directly adding the new
> resource request API to client, we can make scheduler to use it internally to
> make sure it is well defined. And AppSchedulingInfo/SchedulingPlacementSet
> will be the perfect place to isolate which ResourceRequest implementation to
> use.
> 4) Different scheduling requirement needs different behavior of checking
> ResourceRequest table.
> This JIRA is the 1st patch of several refactorings. Which moves all
> ResourceRequest data structure and logics to SchedulingPlacementSet. We need
> follow changes to make it better structured
> - Make delay scheduling to be a plugin of SchedulingPlacementSet
> - After YARN-4902 get committed, change SchedulingPlacementSet to use
> YARN-4902 internally.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
