If flink job recover from previous checkpoint/savepoint, will it re-output records to a different file with same partFileIndex? And can we upgrade flink to 1.4 smoothly?
From: Yun Gao <[email protected]> Date: Friday, October 8, 2021 at 22:43 To: wu shaoj <[email protected]>, [email protected] <[email protected]> Subject: Re: What's the purpose of uniqueId in FileWriterBucket Hi Wu, The uid is used to distinguish between the different subtasks, if removed, the different subtasks of the filesink would have name conflicts if they writes to the same bucket, thus the uid should be necessary if there are multiple subtasks. Best, Yun ------------------Original Mail ------------------ Sender:wu shaoj <[email protected]> Send Date:Fri Oct 8 14:18:34 2021 Recipients:[email protected] <[email protected]> Subject:What's the purpose of uniqueId in FileWriterBucket Hi, folks, >From 1.14, file sink add a uid to a committed file, so would you mind to tell >me what’s the purpose of this field? Can it be removed safely? https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/connectors/datastream/file_sink/#part-file-configuration
