Question regarding end-to-end exactly once guarantee implementation using 2PC?
As I understand how it operates, the pre-phase state is when the checkpoint is initiated and the checkpoint barrier advances from source to sink. Once the pre-phase is complete (and successful), then the next step in the process is where the "Sink" operator is expected to "Commit" the transaction (the data that was part of the checkpointed state). The datasource backed by the Sink operator is expected to commit the transaction. Say if the transaction times out and data cannot be committed, is there an option to roll back the checkpointed state without incurring the data loss? As of now, it does not work this way but I am trying to understand the challenges/limitations with respect to discarding the checkpoint in question? Once reason could be with respect to (what to do with) the subsequent checkpoints that might have advanced during this scenario? Anything else? Regards Vijay -- Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
