Matthias J. Sax created KAFKA-12634:
---------------------------------------
Summary: Should checkpoint after restore finished
Key: KAFKA-12634
URL: https://issues.apache.org/jira/browse/KAFKA-12634
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: Matthias J. Sax
For state stores, Kafka Streams maintains local checkpoint files to track the
offsets of the state store changelog topics. The checkpoint is updated on
commit or when a task is closed cleanly.
However, after a successful restore, the checkpoint is not written. Thus, if an
instance crashes after restore but before committing, even if the state is on
local disk the checkpoint file is missing (indicating that there is no state)
and thus state would be restored from scratch.
While for most cases, the time between restore end and next commit is small,
there are cases when this time could be large, for example if there is no new
input data to be processed (if there is no input data, the commit would be
skipped).
Thus, we should write the checkpoint file after a successful restore to close
this gap (or course, only for at-least-once processing).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
