Kanalas Vidor created KAFKA-13659:
-------------------------------------
Summary: MM2 should read all offset syncs at start up and should
not set consumer offset higher than the end offset
Key: KAFKA-13659
URL: https://issues.apache.org/jira/browse/KAFKA-13659
Project: Kafka
Issue Type: Improvement
Components: mirrormaker
Reporter: Kanalas Vidor
- MirrorCheckpointTask uses OffsetSyncStore, and does not check whether
OffsetSyncStore managed to read to the "end" of the offset-syncs topic.
OffsetSyncStore should fetch the endoffset of the topic at startup, and set a
flag when it finally reaches the endoffset in consumption.
MirrorCheckpointTask.poll should wait for this flag to be true before doing any
in-memory updates and group offset management.
- MirrorCheckpointTask can create checkpoints which point into the "future" -
meaning it sometimes translates consumer offsets in a way that the target
offset is greater than the endoffset of the replica topic partition.
MirrorCheckpointTask should fetch the endoffsets of the affected topics, and
make sure that it does not try to set the consumer offset to anything higher
than the endoffset.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
