Hello, I have a use case where I need to do a cache file deletion after a successful sunk operation(writing to db). My Flink pipeline is built using Java. I am contemplating using Java completableFuture.runasync() to perform the file deletion activity. I am wondering what issues this might cause in terms of thread management and next event processing.
Also related to the same usecase, in another Flink pipeline. I might need to do this cache file deletion in a timed fashion. For example, every five minutes I have to check for cache files that are older than currently opened cache file that is serving some data into the Sink function. All old cache files that are in closed status need to be deleted in a timely manner. All this deletion has to happen asynchronously without blocking the flow of events from Flink source to sink. Also, based on requirement, I cannot make the whole Sink operation async. I have to make the file based cache deletion alone async inside the Sink function. Does Flink support timers or async blocks? Any inputs will be highly helpful. Thanks, JB
