[
https://issues.apache.org/jira/browse/YARN-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362020#comment-15362020
]
Naganarasimha G R commented on YARN-5302:
-----------------------------------------
Hi [~xinxianyin],
Assuming that we aim for YARN-5175, do we require this patch ? IIUC if we do
not use tokens given by user and as per the Vinod's comment *we simply always
manage our own tokens for localization and log-aggregation for long-running
applications / services* i think storing hdfs tokens in statestore might not be
required right ?
cc /[~jianhe]
> Yarn Application log Aggreagation fails due to NM can not get correct HDFS
> delegation token II
> ----------------------------------------------------------------------------------------------
>
> Key: YARN-5302
> URL: https://issues.apache.org/jira/browse/YARN-5302
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Reporter: Xianyin Xin
> Assignee: Xianyin Xin
> Attachments: YARN-5032.001.patch, YARN-5032.002.patch,
> YARN-5302.003.patch, YARN-5302.004.patch
>
>
> Different with YARN-5098, this happens at NM side. When NM recovers,
> credentials are read from NMStateStore. When initialize app aggregators,
> exception happens because of the overdue tokens. The app is a long running
> service.
> {code:title=LogAggregationService.java}
> protected void initAppAggregator(final ApplicationId appId, String user,
> Credentials credentials, ContainerLogsRetentionPolicy
> logRetentionPolicy,
> Map<ApplicationAccessType, String> appAcls,
> LogAggregationContext logAggregationContext) {
> // Get user's FileSystem credentials
> final UserGroupInformation userUgi =
> UserGroupInformation.createRemoteUser(user);
> if (credentials != null) {
> userUgi.addCredentials(credentials);
> }
> ...
> try {
> // Create the app dir
> createAppDir(user, appId, userUgi);
> } catch (Exception e) {
> appLogAggregator.disableLogAggregation();
> if (!(e instanceof YarnRuntimeException)) {
> appDirException = new YarnRuntimeException(e);
> } else {
> appDirException = (YarnRuntimeException)e;
> }
> appLogAggregators.remove(appId);
> closeFileSystems(userUgi);
> throw appDirException;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
