[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15322829#comment-15322829 ]
Inigo Goiri edited comment on YARN-5215 at 6/9/16 4:38 PM: ----------------------------------------------------------- [~jlowe], my view on this feature is more like making the scheduler aware of a "container" that is not managed by YARN and has a dynamic size. Before moving forward with this, we should agree that this is a proper abstraction and complementary to the overcommit effort. Regarding the UI stuff, let's try to make a list of what to expose and I'll give it a try. My first idea is to expose the following per node/queue/cluster: * Node utilization * Container utilization * External utilization * Allocated/unallocated resources Can you think on any other metric/aggregation we should be reporting? was (Author: elgoiri): [~jlowe], my view on this feature is more like making the scheduler our of a "container" that is not managed by YARN. This is a dynamic size container. Before moving forward with this, we should agree that this is a proper abstraction and complementary to the overcommit effort. Regarding the UI stuff, let's try to make a list of what to expose and I'll give it a try. My first idea is to expose the following per node/queue/cluster: * Node utilization * Container utilization * External utilization * Allocated/unallocated resources Can you think on any other metric/aggregation we should be reporting? > Scheduling containers based on external load in the servers > ----------------------------------------------------------- > > Key: YARN-5215 > URL: https://issues.apache.org/jira/browse/YARN-5215 > Project: Hadoop YARN > Issue Type: Improvement > Reporter: Inigo Goiri > Attachments: YARN-5215.000.patch, YARN-5215.001.patch > > > Currently YARN runs containers in the servers assuming that they own all the > resources. The proposal is to use the utilization information in the node and > the containers to estimate how much is consumed by external processes and > schedule based on this estimation. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org