Hi Team, I am using Flink File Source in one of my use case. I observed that, while reading file by source reader it stores its position in checkpointed data. In case application crashes, it restores its position from checkpointed data, once application comes up, which may result in re-emitting few records which were emitted in between last checkpointing and application crash. Whereas in doc link https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/connectors/datastream/guarantees/ I found that File source ensures exactly once delivery semantics with help of data sink. "To guarantee end-to-end exactly-once record delivery (in addition to exactly-once state semantics), the data sink needs to take part in the checkpointing mechanism."
Can someone put some light on this? Regards, Kirti Dhar
