[ https://issues.apache.org/jira/browse/YARN-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774708#comment-16774708 ]
Wilfred Spiegelenburg commented on YARN-8662: --------------------------------------------- Hi [~Sen Zhao], Thank you for filing this and providing a patch. I had some time and finally got around to looking at this for a review. Are you still willing to work on this? It looks like this issue only happen if you use old style resource definitions for the _<maxResources>_ entries. The java doc for {{parseResourceConfigValue}} states: {code} * The {@code missing} parameter is only used in the case of new style * resources without percentages. With new style resources with percentages, * any missing resources will be assumed to be 100% because percentages are * only used with maximum resource limits. {code} Which means that the code is doing what it is documented. You are using old style resources definitions. Your change is going to break this as it will now use the missing parameter also for old style resource definition without percentages. The workaround would be to use the new style declaration and the maximum would be set according to what you would expect. Old style declarations are there for backwards compatibility. When using resource types you really should be using the new style definitions. If we still want to go down this path and make old style behave more like the new style then we have a number of other changes that need to be made: * make a change similar to what you have now * clean up the java doc * clean up user documentation as minimum can take a percentage which is not documented at all * fix the percentage for old style: we need to handle min resources too as now the min for any custom type is 100% of the cluster. If we do not go through we should at least fix the two documentation points and document that you should use the new style definitions for min and max when you use resource types. > Fair Scheduler stops scheduling when a queue is configured only CPU and memory > ------------------------------------------------------------------------------ > > Key: YARN-8662 > URL: https://issues.apache.org/jira/browse/YARN-8662 > Project: Hadoop YARN > Issue Type: Sub-task > Components: fairscheduler > Reporter: Sen Zhao > Assignee: Sen Zhao > Priority: Major > Attachments: NonResourceToSchedule.png, YARN-8662.001.patch > > > Add a new resource type in resource-types.xml, eg: resource1. > In Fair scheduler when queue's MaxResources is configured like: > {code}<maxResources>4096 mb, 4 vcores</maxResources>{code} > When submit a application which need resource like: > {code} 1536 mb, 1 vcores, 10 resource1{code} > The application will be pending. Because there is no resource1 in this queue. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org