Hi Averell, as far as I know these tmp files should be removed when the Flink job is recovering. So you should have these files around only for the latest incomplete checkpoint while recovery has not completed yet.
On Tue, Sep 1, 2020 at 2:56 AM Averell <lvhu...@gmail.com> wrote: > Hello Robert, Arvid, > > As I am running on EMR, and currently AWS only supports version 1.10. > I tried both solutions that you suggested ((i) copying a SAXParser > implementation to the plugins folder and (ii) using the S3FS Plugin from > 1.10.1), and both worked - I could have successful checkpoints. > > However, intermittenly my checkpoints still fail (about 10%). And whenever > it fails, there are non-completed files left in S3 (attached screenshot > below). > I'm not sure whether those uncompleted files are expected, or is that a > bug? > > Thanks and regards, > Averell > < > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t1586/Screen_Shot_2020-08-28_at_11.png> > > > > > -- > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ >
