[
https://issues.apache.org/jira/browse/YARN-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15553303#comment-15553303
]
Eric Payne commented on YARN-2009:
----------------------------------
Thanks, [~sunilg], for your reply.
-----
- {{FifoIntraQueuePreemptionPlugin#calculateIdealAssignedResourcePerApp}}
-- The assignment to {{tmpApp.idealAssigned}} should be cloned:
{code}
tmpApp.idealAssigned = Resources.min(rc, clusterResource,
queueTotalUnassigned, appIdealAssigned);
...
Resources.subtractFrom(queueTotalUnassigned, tmpApp.idealAssigned);
{code}
-- In the above code, if {{queueTotalUnassigned}} is less than
{{appIdealAssigned}}, then {{tmpApp.idealAssigned}} is assigned a reference to
{{queueTotalUnassigned}}. Then, later, {{tmpApp.idealAssigned}} is actually
subtracted from itself.
-----
bq. This current patch will still handle priority and priority + user-limit.
Thoughts?
I am not comfortable with fixing this in another patch. Our main use case is
the one where multiple users need to use the same queue with apps at the same
priority.
- I still need to think through all of the effects, but I was thinking that
something like the following could work:
-- I think my use case is failing because
{{FifoIntraQueuePreemptionPlugin#calculateIdealAssignedResourcePerApp}} orders
the apps by priority. I think that instead, it should order the apps by how
much they are underserved. I think that it should be ordering the apps by
{{tmpApp.toBePreemptedByOther}} instead of priority.
-- Then, if {{calculateIdealAssignedResourcePerApp}} orders the apps by
{{toBePreemptedByOther}}, {{validateOutSameAppPriorityFromDemand}} would also
need to not compare priorities but the app's requirements.
-- I think it should be something like the following, maybe:
{code}
while (lowAppIndex < highAppIndex
&& !apps[lowAppIndex].equals(apps[highAppIndex])
// && apps[lowAppIndex].getPriority() <
apps[highAppIndex].getPriority()) {
&& Resources.lessThan(rc, clusterResource,
apps[lowAppIndex].getToBePreemptFromOther(),
apps[highAppIndex].getToBePreemptFromOther()) ) {
{code}
> Priority support for preemption in ProportionalCapacityPreemptionPolicy
> -----------------------------------------------------------------------
>
> Key: YARN-2009
> URL: https://issues.apache.org/jira/browse/YARN-2009
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
> Reporter: Devaraj K
> Assignee: Sunil G
> Attachments: YARN-2009.0001.patch, YARN-2009.0002.patch,
> YARN-2009.0003.patch, YARN-2009.0004.patch, YARN-2009.0005.patch
>
>
> While preempting containers based on the queue ideal assignment, we may need
> to consider preempting the low priority application containers first.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
