Re: Update UGI with new tokens during the lifespan of a yarn application

2024-06-11 Thread Clay B.
Hi Ankur, There was some work I did in HADOOP-16298; the final code I used for $dayjob works for HDFS and HBase (HBase is non-renewable) tokens but Kubernetes was doing the on-disk token updates for me[1]; I just had to refresh the state in UGI. I ended up making the code proactively refresh

Re: Update UGI with new tokens during the lifespan of a yarn application

2024-06-11 Thread Wei-Chiu Chuang
That sounds like what Spark did. Take a look at this doc https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/security/README.md The Spark AM has a Kerberos keytab and it periodically acquires a new delegation token (the old one is ignored) to make sure it always

Update UGI with new tokens during the lifespan of a yarn application

2024-06-11 Thread Ankur Khanna
Hi experts, I have a use-case with an external session token that is short lived and does not renew(ie, unlike a hadoop delegation token, the expiry time is not updated for this token). For a long running application (longer than the lifespan of the external token), I want to update the UGI/Cre