Re: Communicating between yarn and tasks after delegation token renewal

2022-10-08 Thread Vinod Kumar Vavilapalli
There’s no way to do that. Once YARN launches containers, it doesn’t communicate with them for anything after that. The tasks / containers can obviously always reach out to YARN services. But even that in this case is not helpful because YARN never exposes through APIs what it is doing with

Re: Communicating between yarn and tasks after delegation token renewal

2022-10-07 Thread Julien Phalip
Hi Vinod, Thank you for the quick response. To give you more context, I would like to reduce the load on the remote target service. My idea was that, when YARN renews the token, the remote service would also return an updated piece of data to YARN. Then YARN would somehow share that piece of

Communicating between yarn and tasks after delegation token renewal

2022-10-07 Thread Julien Phalip
Hi, IIUC, when a distributed job is started, Yarn first obtains a delegation token from the target resource, then securely pushes the delegation token to the individual tasks. If the job lasts longer than a given period of time, then Yarn renews the delegation token (or more precisely, extends