Hi Ankur,
There was some work I did in HADOOP-16298; the final code I used for
$dayjob works for HDFS and HBase (HBase is non-renewable) tokens but
Kubernetes was doing the on-disk token updates for me[1]; I just had to
refresh the state in UGI. I ended up making the code proactively refresh
tokens rather than wait for a token error, as there was a race condition
when HBase 2.x came out. I think my last copy as I was trying to port it
upstream was at:
https://github.com/cbaenziger/hadoop/tree/hadoop-16298-wip (however, I no
longer recall what was left to do).
Unfortunately, I got moved off this for my day job so haven't had time to
revisit completing the contribution. Perhaps some of my hacks can help
you? Also the testing got a bit zaney[2] to try and tickle races.
-Clay
[1]: A shim to get tokens similarly can be seen at
https://github.com/cbaenziger/hadoop-token-cli
[2]: The testing code I used is at
https://github.com/cbaenziger/delegation_token_tests
On Tue, 11 Jun 2024, Wei-Chiu Chuang wrote:
That sounds like what Spark did.Take a look at this
doc
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/security/README.md
The Spark AM has a Kerberos keytab and it periodically acquires a new
delegation token (the old one is ignored) to make sure it always has a valid DT.
Finally, distribute the DT to all executors.
On Tue, Jun 11, 2024 at 4:34 AM Ankur Khanna
wrote:
Hi experts,
I have a use-case with an external session token that is short lived and
does not renew(ie, unlike a hadoop delegation token, the expiry time
is not updated for this token). For a long running application (longer
than the lifespan of the external token), I want to update the
UGI/Credential object of each and every worker container with a new token.
If I understand correctly, all delegation tokens are shared at the launch
of a container.
Is there any way to update the credential object after the launch of the
container and during the lifespan of the application?
Best,
Ankur Khanna
-
To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org
For additional commands, e-mail: user-h...@hadoop.apache.org
