RE: monitor finished files on a Continues Reader

2019-05-20 Thread Hanan Yehudai
It helps ! thank you  From: Aljoscha Krettek Sent: 20 May 2019 12:45 To: Hanan Yehudai Cc: user@flink.apache.org Subject: Re: monitor finished files on a Continues Reader Hi, I think what you’re trying to achieve is not possible with the out-of-box file source. The problem

Re: monitor finished files on a Continues Reader

2019-05-20 Thread Aljoscha Krettek
Hi, I think what you’re trying to achieve is not possible with the out-of-box file source. The problem is that it is hard to know when a file can be deleted, i.e. there are multiple splits of a file and those are possibly read on different parallel operators. Plus, deletion/move of files has

monitor finished files on a Continues Reader

2019-05-20 Thread Hanan Yehudai
Hi im looking for a way to delete / rename files that are done loading.. im using the env.readFile , monitoring a directory for all new files, once files are done with I would like to delete it. Is there a way to monitor the closed splits in the continues reader ? is there an different way