[
https://issues.apache.org/jira/browse/YARN-2314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171420#comment-14171420
]
Jason Lowe commented on YARN-2314:
----------------------------------
Yes, the patch sets the default to off since that allows all cluster sizes to
work. If it's crucial to default to enabled for small clusters then those with
large clusters will have to manually configure the cache off. Again I have yet
to see evidence this cache is necessary, so defaulting to something that
doesn't fail for all cluster sizes seemed like a better choice than one which
would work for some but not others. If you have evidence where Tez absolutely
has to have this cache enabled that would be good to share.
> ContainerManagementProtocolProxy can create thousands of threads for a large
> cluster
> ------------------------------------------------------------------------------------
>
> Key: YARN-2314
> URL: https://issues.apache.org/jira/browse/YARN-2314
> Project: Hadoop YARN
> Issue Type: Bug
> Components: client
> Affects Versions: 2.1.0-beta
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Priority: Critical
> Attachments: YARN-2314.patch, disable-cm-proxy-cache.patch,
> nmproxycachefix.prototype.patch
>
>
> ContainerManagementProtocolProxy has a cache of NM proxies, and the size of
> this cache is configurable. However the cache can grow far beyond the
> configured size when running on a large cluster and blow AM address/container
> limits. More details in the first comment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
