Hi Dan,

The file sink would first write records to temporary files,
namely .part-*, and commit them on checkpoint succeeding
by renaming them to formal files, namely part-*.

Best,
Yun


------------------------------------------------------------------
From:Dan Hill <quietgol...@gmail.com>
Send Time:2021 Dec. 1 (Wed.) 13:51
To:user <user@flink.apache.org>
Subject:Does Flink ever delete any sink S3 files?

Hi.

I'm debugging an issue with a system that listens for files written to S3.  I'm 
assuming Flink does not modify sink objects after they've been written.  I've 
seen a minicluster test locally write a ".part-" file.  I wanted to double 
check to make sure S3 sinks won't write and delete files.

```FileSink.forBulkFormat(new Path(...), factory)
        .withBucketAssigner(new DateHourBucketAssigner<...>(...))
        .withRollingPolicy(OnCheckpointRollingPolicy.build())
        .build())
    .uid("sink-s3")
    .name("S3 sink");
```


Reply via email to