[no subject]

Hi , Is it possible for an executor (or slave) to know when an actual job ends? I'm running spark on a cluster (with yarn) and my workers create some temporary files that I would like to clean up once the job ends. Is there a way for the worker to detect that a job has finished? I tried doing it i

Is it possible to get a job end kind of notification on the executor (slave)

Hi , Is it possible for an executor (or slave) to know when an actual job ends? I'm running spark on a cluster (with yarn) and my workers create some temporary files that I would like to clean up once the job ends. Is there a way for the worker to detect that a job has finished? I tried doing it i