Thanks for the responses and thanks Gary for the confirmation. Just to give some background, we deploy Flink inside Kubernetes so there is a chance that TaskManagers COULD be shut down in a non-graceful way leaving cache artifacts on the temporary volumes.
With Gary's confirmation, we'll add an init container to make sure the volumes are cleared before a TM starts. /David/ On Thu, Mar 12, 2020 at 8:24 AM Gary Yao <g...@apache.org> wrote: > Hi David, > > > Would it be safe to automatically clear the temporary storage every time > when a TaskManager is started? > > (Note: the temporary volumes in use are dedicated to the TaskManager and > not shared :-) > Yes, it is safe in your case. > > Best, > Gary > > On Tue, Mar 10, 2020 at 6:39 PM David Maddison <maddisonda...@gmail.com> > wrote: > >> Hi, >> >> When a TaskManager is restarted it can leave behind unreferenced >> BlobServer cache directories in the temporary storage that never get >> cleaned up. Would it be safe to automatically clear the temporary storage >> every time when a TaskManager is started? >> >> (Note: the temporary volumes in use are dedicated to the TaskManager and >> not shared :-) >> >> Thanks in advance, >> >> David. >> >
