[
https://issues.apache.org/jira/browse/TEZ-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506317#comment-16506317
]
Sergey Shelukhin commented on TEZ-3904:
---------------------------------------
Yeah that's the idea, although in the case of Tez the actual renewer may not
live fully in Tez (since some tokens, like the ones for HBase/etc., are
originally obtained by Hive, and some are obtained by Tez based on paths).
Might make sense to allow the users of Tez (ie Hive) to supply a function to
get the former instead of the tokens themselves, so Tez could get the new
tokens at any time.
The containers would also need to get tokens.
> an API to update tokens for Tez AM and the DAG
> ----------------------------------------------
>
> Key: TEZ-3904
> URL: https://issues.apache.org/jira/browse/TEZ-3904
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Sergey Shelukhin
> Priority: Major
>
> Nothing is permanent in this world, lest of all delegation tokens.
> The current way around token expiration (the one where you cannot keep
> renewing anymore) in Hive when Tez AM is used in session mode is to cycle Tez
> AM. It may happen though that a query is running at that time, and so the AM
> cannot be restarted with new tokens. We let the query run its course and it
> usually dies because it tries to do something with an expired token.
> To get around that, we cycle AMs a few hours before tokens are going to
> expire.
> However, that is still not ideal because it puts an upper bound on safe Hive
> query runtime (a query longer than 3 hours with current config may fail due
> to an expired token if its timing is unlucky), and also precludes setting
> tokens to expire much faster than the standard 7-day time frame.
> There should be a mechanism to replace tokens in the AM, including for a
> running DAG.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
