[
https://issues.apache.org/jira/browse/YARN-6523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16366935#comment-16366935
]
Manikandan R edited comment on YARN-6523 at 2/16/18 12:48 PM:
--------------------------------------------------------------
Thank you [~rohithsharma] for helping me in setting up a secure cluster. Huh !
:) Thank you [~bibinchundatt] for inputs.
[~Naganarasimha] / [~jlowe]
Was able to test the patch in live pseudo cluster and able to see token
sequence no getting incremented as and when any new token fetches from HDFS.
Can you please review the patch?
was (Author: maniraj...@gmail.com):
Thank you [~rohithsharma] for helping me in setting up a secure cluster. Huh !
:) Thank you [~bibinchundatt] for inputs.
[~Naganarasimha] / [~jlowe]
Was able to test the patch in live pseudo cluste and able to see token sequence
no getting incremented as and when any new token fetches from HDFS. Can you
please review the patch?
> Newly retrieved security Tokens are sent as part of each heartbeat to each
> node from RM which is not desirable in large cluster
> -------------------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-6523
> URL: https://issues.apache.org/jira/browse/YARN-6523
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: RM
> Affects Versions: 2.8.0, 2.7.3
> Reporter: Naganarasimha G R
> Assignee: Manikandan R
> Priority: Major
> Attachments: YARN-6523.001.patch, YARN-6523.002.patch
>
>
> Currently as part of heartbeat response RM sets all application's tokens
> though all applications might not be active on the node. On top of it
> NodeHeartbeatResponsePBImpl converts tokens for each app into
> SystemCredentialsForAppsProto. Hence for each node and each heartbeat too
> many SystemCredentialsForAppsProto objects were getting created.
> We hit a OOM while testing for 2000 concurrent apps on 500 nodes cluster with
> 8GB RAM configured for RM
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
