[
https://issues.apache.org/jira/browse/HDFS-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028002#comment-14028002
]
Chris Nauroth commented on HDFS-6222:
-------------------------------------
Hi, [~daryn]. Can you describe what kinds of problems you've seen "for
long-running daemons"? Is there a race condition where the renewer thread
might not do the renewal ahead of the client operation that needs it?
Something else?
A couple of questions/observations:
# I have a question on this logic:
{code}
} else {
token = getDelegationToken(null);
if (token != null) {
LOG.debug("Fetched new token: " + token);
} else { // security is disabled
canRefreshDelegationToken = false;
}
}
{code}
In the second else block, I don't see how we can reach this code path when
security is disabled, because it's already guarded by an earlier check of
{{canRefreshDelegationToken}}, which is initialized from
{{UserGroupInformation#isSecurityEnabled}}. Also, I seem to recall you
mentioned to me offline seeing certain failures where a malformed response from
the server can cause the JSON parser to return a null object. If that happens,
then this logic would turn off future attempts to refresh. Would that
effectively leave the file system instance in an unusable state, with the only
workaround being to discard the instance and get a new one?
# It's unfortunate that we have to parse the exception strings, but I don't see
an alternative. Do you know if these exception messages are consistent across
prior versions? Is there any chance that a 2.5.0 {{WebHdfsFileSystem}}
instance would be incompatible with servers running earlier versions back to
2.2.0?
# The latency of token renewal is now synchronous with the client operation.
It's probably no big deal if your internal testing on secure clusters didn't
show a problem.
# Unless I'm misreading, it seems like much of {{TokenAspect}} has become dead
code. Is there a need for further clean-up?
# Thanks for adding those tests. They're very thorough. Do you think we need
to add assertions that we're getting the correct token kind? Is it also
worthwhile to add some tests through swebhdfs too, and assert that they get
{{SWebHdfsFileSystem#TOKEN_KIND}}?
> Remove background token renewer from webhdfs
> --------------------------------------------
>
> Key: HDFS-6222
> URL: https://issues.apache.org/jira/browse/HDFS-6222
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
> Affects Versions: 2.0.0-alpha, 3.0.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: HDFS-6222.branch-2.patch, HDFS-6222.branch-2.patch,
> HDFS-6222.trunk.patch, HDFS-6222.trunk.patch
>
>
> The background token renewer is a source of problems for long-running
> daemons. Webhdfs should lazy fetch a new token when it receives an
> InvalidToken exception.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
