junaiddshaukat opened a new issue, #39634: URL: https://github.com/apache/beam/issues/39634
The Kafka Streams runner stores an unbounded source's checkpoint mark so a restart resumes from it, but never calls `CheckpointMark.finalizeCheckpoint()`. Sources that rely on finalization to acknowledge or release data will not see it. A mark should only be finalized once it is durably committed, which needs a hook that runs before Kafka Streams commits. The runner does not have one — the same gap that prevents bundles being bounded by time. A related point from review: a bounded split should likewise only be committed after it has been completely processed and flushed downstream. Part of #18479. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
