Hello Experts,
we have a flink streaming job which read data from kafka and sink it to S3. We
were using Flink's internal table API to achieve this. and I have a case where
I am not able to recover my job from snapshot.
I am getting below exception. Can you please help.
2021-10-08 18:43:13
java.io.IOException: Recovering commit failed for object
topics/testtopic/YEAR=2021/MONTH=10/DAY=08/HOUR=21/MIN=34/.uncompacted-part-3ae64209-a2b5-4f42-8f70-1e46e7e0a83c-1-2.
Object does not exist and MultiPart Upload
JsbsI_mYvHTkB2GgPpuyfD49JuEfGjtjQ6I6LQJ0CZMEI0XCcpcVtMLIWCU635jvh0nOlbNrqhmfSjUdc2hrEId.eqP2P93O44s5rBEUZWwehcfzrIlQb.U0GfZeb1.wOp2tkD5TTNzd8waWBOr7sw--
is not valid.
at
org.apache.flink.fs.s3.common.writer.S3Committer.commitAfterRecovery(S3Committer.java:123)
at
org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter$OutputStreamBasedPendingFile.commitAfterRecovery(OutputStreamBasedPartFileWriter.java:218)
at
org.apache.flink.streaming.api.functions.sink.filesystem.Bucket.commitRecoveredPendingFiles(Bucket.java:160)
at
org.apache.flink.streaming.api.functions.sink.filesystem.Bucket.<init>(Bucket.java:127)
at
org.apache.flink.streaming.api.functions.sink.filesystem.Bucket.restore(Bucket.java:466)
at org.apache.flink.streaming.api.functions.sink.filesystem.Def
regards,
Shamit Jain
