[
https://issues.apache.org/jira/browse/YARN-9785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918310#comment-16918310
]
Sunil Govindan edited comment on YARN-9785 at 8/29/19 6:05 AM:
---------------------------------------------------------------
Thanks [~bibinchundatt] for helping to clarify the issues. As far as we
discussed, we are dealing with TWO separate issues here.
# When a new resource type is added and no resources are configured for that
resource type, then we have 0 in the resource entry. This causes the comparison
issues. With the wip patch, we can resolve this pblm.
# However as you pointed out in the 3rd case in above comment , Dominant
resource calculator will kick in as CPU is more on RHS and Memory is more on
LHS. But in ratio level, RHS has upper hand and CPU will become dominant here.
This literally messes up the comparison of AM limi in Leaf Queue. As you
mentioned, this can be fixed by changing to use fitsIn instead of
lessThanOrEquals method.
Could we create a new Jira to track issue #2 mentioned here for AMLimit and
this current issue can track the GPU resource 0 issue alone. cc [~leftnoteasy]
Thanks
was (Author: sunilg):
Thanks [~bibinchundatt] for helping to clarify the issues. As far as we
discussed, we are dealing with TWO separate issues here.
# When a new resource type is added and no resources are configured for that
resource type, then we have 0 in the resource entry. This causes the comparison
issues. With the wip patch, we can resolve this pblm.
# However as you pointed out in the 3rd case in above comment , Dominant
resource calculator will kick in as CPU is more on RHS and Memory is more on
LHS. But in ratio level, RHS has upper hand and CPU will become dominant here.
This literally messes up the comparison of AM limi in Leaf Queue. As you
mentioned, this can be fixed by changing to use fitsIn instead of
lessThanOrEquals method.
Could we create a new Jira to track issue #2 mentioned here for AMLimit and
this current issue can track the GPU resource 0 issue alone.
Thanks
> Fix DominantResourceCalculator when one resource is zero
> --------------------------------------------------------
>
> Key: YARN-9785
> URL: https://issues.apache.org/jira/browse/YARN-9785
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Bilwa S T
> Assignee: Bilwa S T
> Priority: Blocker
> Attachments: YARN-9785-001.patch, YARN-9785.wip.patch
>
>
> Configure below property in resource-types.xml
> {quote}<property>
> <name>yarn.resource-types</name>
> <value>yarn.io/gpu</value>
> </property>
> {quote}
> Submit applications even after AM limit for a queue is reached. Applications
> get activated even after limit is reached
> !queue.png!
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
