Hi Harry, Thanks for your question!
There were negative lag/data loss bugs fixed some time back [1, 2] so check what version of the MirrorMaker connectors you're using. I would recommend using the latest version of MM2 (3.8.0) if possible. After upgrading you will need to clear the checkpoints topic and target consumer offsets to allow MM2 to translate the offsets again, as MM2 will not rewind the target consumer group in situations like this. Thanks, and hope this helps! Greg [1] https://issues.apache.org/jira/browse/KAFKA-12635 [2] https://issues.apache.org/jira/browse/KAFKA-12468 On Fri, Aug 23, 2024 at 5:24 PM Harry Fallows <harryfall...@protonmail.com.invalid> wrote: > Hi everyone, > > Does anyone know if it is possible for the Checkpoint connector to be > ahead of the Source connector? > > For context, I have a target Kafka cluster that is reporting negative lag > because the log end offset is 0 but the consumer group offset is 100. I > have a tenuous theory that this could be caused in the followng sequence of > events: > > - Offset is translated. > - Messages in topic are cleaned up before being replicated. > > Does this sound possible? Any insight would be greatly appreciated. > > Kind regards > Harry
