[
https://issues.apache.org/jira/browse/YARN-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774708#comment-16774708
]
Wilfred Spiegelenburg commented on YARN-8662:
---------------------------------------------
Hi [~Sen Zhao], Thank you for filing this and providing a patch.
I had some time and finally got around to looking at this for a review. Are you
still willing to work on this?
It looks like this issue only happen if you use old style resource definitions
for the _<maxResources>_ entries.
The java doc for {{parseResourceConfigValue}} states:
{code}
* The {@code missing} parameter is only used in the case of new style
* resources without percentages. With new style resources with percentages,
* any missing resources will be assumed to be 100% because percentages are
* only used with maximum resource limits.
{code}
Which means that the code is doing what it is documented. You are using old
style resources definitions. Your change is going to break this as it will now
use the missing parameter also for old style resource definition without
percentages.
The workaround would be to use the new style declaration and the maximum would
be set according to what you would expect. Old style declarations are there for
backwards compatibility. When using resource types you really should be using
the new style definitions.
If we still want to go down this path and make old style behave more like the
new style then we have a number of other changes that need to be made:
* make a change similar to what you have now
* clean up the java doc
* clean up user documentation as minimum can take a percentage which is not
documented at all
* fix the percentage for old style: we need to handle min resources too as now
the min for any custom type is 100% of the cluster.
If we do not go through we should at least fix the two documentation points and
document that you should use the new style definitions for min and max when you
use resource types.
> Fair Scheduler stops scheduling when a queue is configured only CPU and memory
> ------------------------------------------------------------------------------
>
> Key: YARN-8662
> URL: https://issues.apache.org/jira/browse/YARN-8662
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: fairscheduler
> Reporter: Sen Zhao
> Assignee: Sen Zhao
> Priority: Major
> Attachments: NonResourceToSchedule.png, YARN-8662.001.patch
>
>
> Add a new resource type in resource-types.xml, eg: resource1.
> In Fair scheduler when queue's MaxResources is configured like:
> {code}<maxResources>4096 mb, 4 vcores</maxResources>{code}
> When submit a application which need resource like:
> {code} 1536 mb, 1 vcores, 10 resource1{code}
> The application will be pending. Because there is no resource1 in this queue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
