Hi ,
Is it possible for an executor (or slave) to know when an actual job ends?
I'm running spark on a cluster (with yarn) and my workers create some
temporary files that I would like to clean up once the job ends. Is there a
way for the worker to detect that a job has finished? I tried doing it i
Hi ,
Is it possible for an executor (or slave) to know when an actual job ends?
I'm running spark on a cluster (with yarn) and my workers create some
temporary files that I would like to clean up once the job ends. Is there a
way for the worker to detect that a job has finished? I tried doing it i