[ https://issues.apache.org/jira/browse/HDFS-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14188293#comment-14188293 ]
Steve Loughran commented on HDFS-7295: -------------------------------------- ~bcwalrus bq. > pushing out new tokens from the client bq. When the Spark Streaming app is running, it's all in the cluster. It doesn't have any Kerberos credential at that point. I don't think it can get new tokens. Right? the authenticated user can themselves create then push out new tokens from outside the cluster. If an app has been deployed by management tooling, the tooling it can schedule the refesh. The deployed app needs to export a (secured) IPC channel to received token updates in the AM and propagate them to its containers. AFAIK, this is what Twill does. I've been working with my colleagues on long-lived YARN services for over a year now; token expiry is one piece of complexity you hit. The YARN team have addressed the RM/AM token problem, as well as the AM-launch-context token problem. For the rest, they've said "we will help you get your keytabs over". HBase being deployed under YARN needs the same keytabs as HBase-classic; here we actually gain from having a keytab propagation mechanism. For us we also have to add the keytab support to the AM —but it is needed for more than just HDFS. As an example: we also need authenticated write access to ZK when we need to update the YARN registry. Once you deploy applications that start needing to talk to other YARN services then the auth problem crops up there too. Turning off token expiry for HDFS doesn't help there anyway —all it does is set a dangerous precedent for the rest of the infrastructure. I guess you are disappointed by the negative feedback here: you had a simple solution to the problem of HDFS token expiry without having to distribute keytabs. It's just that it is at odds with the goals of kerberized-Hadoop: a secure cluster. This is precisely why those of us who have been working on long lived YARN services for the past year haven't already actually implemented it. Sorry. To summarise: h3. This is not the solution you are looking for > Support arbitrary max expiration times for delegation token > ----------------------------------------------------------- > > Key: HDFS-7295 > URL: https://issues.apache.org/jira/browse/HDFS-7295 > Project: Hadoop HDFS > Issue Type: Improvement > Reporter: Anubhav Dhoot > Assignee: Anubhav Dhoot > > Currently the max lifetime of HDFS delegation tokens is hardcoded to 7 days. > This is a problem for different users of HDFS such as long running YARN apps. > Users should be allowed to optionally specify max lifetime for their tokens. -- This message was sent by Atlassian JIRA (v6.3.4#6332)