[ https://issues.apache.org/jira/browse/YARN-2314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131732#comment-14131732 ]
Lohit Vijayarenu commented on YARN-2314: ---------------------------------------- We hit same problem on one of our large cluster with more than 2.5K nodes. As a work around we ended up increasing container size to 6G for AM (and with pmem-vmem ratio of 2:1) we give away 12G of VM for AM container. From initial looks of this, there is no way to turn this behavior off via config, other than patching code, right? > ContainerManagementProtocolProxy can create thousands of threads for a large > cluster > ------------------------------------------------------------------------------------ > > Key: YARN-2314 > URL: https://issues.apache.org/jira/browse/YARN-2314 > Project: Hadoop YARN > Issue Type: Bug > Components: client > Affects Versions: 2.1.0-beta > Reporter: Jason Lowe > Priority: Critical > Attachments: nmproxycachefix.prototype.patch > > > ContainerManagementProtocolProxy has a cache of NM proxies, and the size of > this cache is configurable. However the cache can grow far beyond the > configured size when running on a large cluster and blow AM address/container > limits. More details in the first comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)