[
https://issues.apache.org/jira/browse/HDFS-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14188293#comment-14188293
]
Steve Loughran commented on HDFS-7295:
--------------------------------------
~bcwalrus
bq. > pushing out new tokens from the client
bq. When the Spark Streaming app is running, it's all in the cluster. It
doesn't have any Kerberos credential at that point. I don't think it can get
new tokens. Right?
the authenticated user can themselves create then push out new tokens from
outside the cluster. If an app has been deployed by management tooling, the
tooling it can schedule the refesh. The deployed app needs to export a
(secured) IPC channel to received token updates in the AM and propagate them to
its containers. AFAIK, this is what Twill does.
I've been working with my colleagues on long-lived YARN services for over a
year now; token expiry is one piece of complexity you hit. The YARN team have
addressed the RM/AM token problem, as well as the AM-launch-context token
problem. For the rest, they've said "we will help you get your keytabs over".
HBase being deployed under YARN needs the same keytabs as HBase-classic; here
we actually gain from having a keytab propagation mechanism. For us we also
have to add the keytab support to the AM —but it is needed for more than just
HDFS. As an example: we also need authenticated write access to ZK when we need
to update the YARN registry. Once you deploy applications that start needing
to talk to other YARN services then the auth problem crops up there too.
Turning off token expiry for HDFS doesn't help there anyway —all it does is set
a dangerous precedent for the rest of the infrastructure.
I guess you are disappointed by the negative feedback here: you had a simple
solution to the problem of HDFS token expiry without having to distribute
keytabs. It's just that it is at odds with the goals of kerberized-Hadoop: a
secure cluster. This is precisely why those of us who have been working on long
lived YARN services for the past year haven't already actually implemented it.
Sorry.
To summarise:
h3. This is not the solution you are looking for
> Support arbitrary max expiration times for delegation token
> -----------------------------------------------------------
>
> Key: HDFS-7295
> URL: https://issues.apache.org/jira/browse/HDFS-7295
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Anubhav Dhoot
> Assignee: Anubhav Dhoot
>
> Currently the max lifetime of HDFS delegation tokens is hardcoded to 7 days.
> This is a problem for different users of HDFS such as long running YARN apps.
> Users should be allowed to optionally specify max lifetime for their tokens.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
