Sergey Shelukhin created TEZ-3904:
-------------------------------------
Summary: an API to update tokens for Tez AM and the DAG
Key: TEZ-3904
URL: https://issues.apache.org/jira/browse/TEZ-3904
Project: Apache Tez
Issue Type: Bug
Reporter: Sergey Shelukhin
Nothing is permanent in this world, lest of all delegation tokens.
The current way around token expiration (the one where you cannot keep renewing
anymore) in Hive when Tez AM is used in session mode is to cycle Tez AM. It may
happen though that a query is running at that time, and so the AM cannot be
restarted with new tokens. We let the query run its course and it usually dies
because it tries to do something with an expired token.
To get around that, we cycle AMs a few hours before tokens are going to expire.
However, that is still not ideal because it puts an upper bound on safe Hive
query runtime (a query longer than 3 hours with current config may fail due to
an expired token if its timing is unlucky), and also precludes setting tokens
to expire much faster than the standard 7-day time frame.
There should be a mechanism to replace tokens in the AM, including for a
running DAG.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
