[jira] [Commented] (YARN-2555) Effective max-allocation-* should consider biggest node
[ https://issues.apache.org/jira/browse/YARN-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135122#comment-14135122 ] Wangda Tan commented on YARN-2555: -- As pointed by [~vinodkv], I think YARN-394 created a clear criteria about what RR should be rejected and what not when it cannot be met. That make sense to me. > Effective max-allocation-* should consider biggest node > --- > > Key: YARN-2555 > URL: https://issues.apache.org/jira/browse/YARN-2555 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 2.5.1 >Reporter: Karthik Kambatla > > The effective max-allocation-mb should be > min(admin-configured-max-allocation-mb, max-mb-on-one-node), so we can reject > container requests for resources larger than any node. Today, these requests > wait forever. > We should do this for all resources and update the effective value on node > updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2555) Effective max-allocation-* should consider biggest node
[ https://issues.apache.org/jira/browse/YARN-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135013#comment-14135013 ] Sandy Ryza commented on YARN-2555: -- [~gp.leftnoteasy], this isn't the same as having an NM variable affect the RM conf. Considering the effective max allocation as the biggest node means rejecting requests that won't fit on any node, which I believe is the correct behavior. The issue I had with YARN-2422 was handling at this at the configuration level, rather than properly handling this for heterogeneous clusters. Thanks for pointing that out [~agentvindo.dev] - agreed that this duplicates YARN-56. I think something like the approach outlined here probably makes the most sense for that JIRA. > Effective max-allocation-* should consider biggest node > --- > > Key: YARN-2555 > URL: https://issues.apache.org/jira/browse/YARN-2555 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 2.5.1 >Reporter: Karthik Kambatla > > The effective max-allocation-mb should be > min(admin-configured-max-allocation-mb, max-mb-on-one-node), so we can reject > container requests for resources larger than any node. Today, these requests > wait forever. > We should do this for all resources and update the effective value on node > updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2555) Effective max-allocation-* should consider biggest node
[ https://issues.apache.org/jira/browse/YARN-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14134868#comment-14134868 ] Wangda Tan commented on YARN-2555: -- I think they're different, proposal of YARN-2422 is making a flexible max-allocation. And this JIRA focus on reject ResourceRequest when its larger than biggest node in the cluster. IMHO, we don't need to both of them, it is very possible nodes connect to RM after application submitted, especially in virtual cluster environment. And as [~sandyr] commented, it's weird to have NM variable affect RM configuration. Having a fixed max-allocation is useful to make sure user doesn't get more resource than he needed. > Effective max-allocation-* should consider biggest node > --- > > Key: YARN-2555 > URL: https://issues.apache.org/jira/browse/YARN-2555 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 2.5.1 >Reporter: Karthik Kambatla >Assignee: Wei Yan > > The effective max-allocation-mb should be > min(admin-configured-max-allocation-mb, max-mb-on-one-node), so we can reject > container requests for resources larger than any node. Today, these requests > wait forever. > We should do this for all resources and update the effective value on node > updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2555) Effective max-allocation-* should consider biggest node
[ https://issues.apache.org/jira/browse/YARN-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14134854#comment-14134854 ] Varun Vasudev commented on YARN-2555: - Duplicate of YARN-2422? > Effective max-allocation-* should consider biggest node > --- > > Key: YARN-2555 > URL: https://issues.apache.org/jira/browse/YARN-2555 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 2.5.1 >Reporter: Karthik Kambatla >Assignee: Wei Yan > > The effective max-allocation-mb should be > min(admin-configured-max-allocation-mb, max-mb-on-one-node), so we can reject > container requests for resources larger than any node. Today, these requests > wait forever. > We should do this for all resources and update the effective value on node > updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
