[ https://issues.apache.org/jira/browse/YARN-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15287660#comment-15287660 ]
Karthik Kambatla commented on YARN-4866: ---------------------------------------- Thanks for working on this, Yufei. Comments on the patch: # In addition to checking the cluster resources, should we also check against queue's max resources for this queue and any parents? For the latter, a helper method might be needed. # Nit: Can we avoid naming the variables with a "not" for readability? For instance, may be we could use overMaxAMShareLimit instead of notOverLimit and exhaustsVcores instead of notTakeAllVCore and negate the expressions as necessary? # Nit: Also, do we really need the second boolean. How about modifying the newly added if condition to if (!overMaxAMShareLimit && policy is not DRF) and just override overMaxAMShareLimit? The method itself could return !overMaxAMShareLimit? # Nit: After the changes, if the code needs more explanation, should we add a comment before the if condition so it is clear why the second check? > FairScheduler: AMs can consume all vcores leading to a livelock when using > FAIR policy > -------------------------------------------------------------------------------------- > > Key: YARN-4866 > URL: https://issues.apache.org/jira/browse/YARN-4866 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler > Affects Versions: 2.8.0 > Reporter: Karthik Kambatla > Assignee: Yufei Gu > Attachments: YARN-4866.001.patch, YARN-4866.002.patch, > YARN-4866.003.patch, YARN-4866.004.patch > > > The maxAMShare uses the queue's policy for enforcing limits. When using FAIR > policy, this considers only memory. If there are fewer vcores on the cluster, > the AMs can end up taking all the vcores leading to a livelock. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org