[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16624224#comment-16624224 ]
Arun Suresh edited comment on YARN-8808 at 9/21/18 9:50 PM: ------------------------------------------------------------ While working on YARN-1013, I check also for whether the aggregateUtilization/nodeUtilization == 0. This implies that nothing is running on the node, which implies that we should not over-allocate on the node right ? Also, I am thinking a combination of containerUtilization + nodeUtilization should be used though. Consider the situation where the container utilization is high but the node utilization is low - Node has capacity for 4 1GB containers, but is currently running 2 containers each using more than 1.9GB - in this case, overallocation should be allowed. was (Author: asuresh): While working on YARN-1013, looks like we should also check for whether the aggregateUtilization/nodeUtilization == 0. This implies that nothing is running on the node, which implies that we should not over-allocate on the node right ? Also, I am thinking a combination of containerUtilization + nodeUtilization should be used though. Consider the situation where the container utilization is high but the node utilization is low - Node has capacity for 4 1GB containers, but is currently running 2 containers each using more than 1.9GB - in this case, overallocation should be allowed. > Use aggregate container utilization instead of node utilization to determine > resources available for oversubscription > --------------------------------------------------------------------------------------------------------------------- > > Key: YARN-8808 > URL: https://issues.apache.org/jira/browse/YARN-8808 > Project: Hadoop YARN > Issue Type: Sub-task > Affects Versions: YARN-1011 > Reporter: Haibo Chen > Assignee: Haibo Chen > Priority: Major > Attachments: YARN-8088-YARN-1011.01.patch, > YARN-8808-YARN-1011.00.patch > > > Resource oversubscription should be bound to the amount of the resources that > can be allocated to containers, hence the allocation threshold should be with > respect to aggregate container utilization. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org