2017-12-08 18:25 GMT+08:00 Stefan Richter <s.rich...@data-artisans.com>: > You need to be a bit careful if your sink needs exactly-once semantics. In > this case things should either be idempotent or the db must support rolling > back changes between checkpoints, e.g. via transactions. Commits should be > triggered for confirmed checkpoints („notifyCheckpointComplete“).
I doubt if we have a risk here: in notifyCheckpointComplete, the checkpoint was completed, and if the process crashes (or machine failure) before it commits the db, the flink would restart the app, restoring the state from the last checkpoint, but it would not invoke notifyCheckpointComplete again? correct? if so, we would miss the database ingress for the data between the last two checkpoints, am I correct?
