Hi Francis, A part of every complete snapshot is the record positions associated with the barrier that triggered the checkpointing of this snapshot. The snapshot is completed only when all the records within the checkpoint reaches the sink. When a topology fails, all the operators' state will fall back to the latest complete snapshot (incomplete snapshots will be ignored). The data source will also fall back to the position recorded with this snapshot, so even if there are repeatedly read data records after the restore, the restored operator's state are also clean of the records effect. This way, Flink guarantees exactly-once effects of each record on every operator's state. The user functions in operators need not to be implemented idempotent.
Hope this helps answer your question! Cheers, Gordon -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Checkpoint-for-exact-once-proccessing-tp4261p4264.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
