[
https://issues.apache.org/jira/browse/HBASE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15220236#comment-15220236
]
Gary Helmling commented on HBASE-15570:
---------------------------------------
The way we dealt with this issue in Apache Twill, which also supports
"run-forever" applications on YARN, is to have the controller process which
submits the application (which also has kerberos credentials) periodically
request new delegation tokens for each of the needed services and push these
down to the running containers.
How were you planning to tackle this? Would the renewer have kerberos
credentials? Would we start tracking token expiration times on the server side
or would updated tokens be issued?
> renewable delegation tokens for long-lived spark applications
> -------------------------------------------------------------
>
> Key: HBASE-15570
> URL: https://issues.apache.org/jira/browse/HBASE-15570
> Project: HBase
> Issue Type: Improvement
> Components: spark
> Reporter: Sean Busbey
> Assignee: Sean Busbey
>
> Right now our spark integration works on secure clusters by getting
> delegation tokens and sending them to the executors. Unfortunately,
> applications that need to run for longer than the delegation token lifetime
> (by default 7 days) will fail.
> In particular, this is an issue for Spark Streaming applications. Since they
> expect to run indefinitely, we should have a means for renewing the
> delegation tokens.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
