[ https://issues.apache.org/jira/browse/KAFKA-7500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929057#comment-16929057 ]
Christian Hagel commented on KAFKA-7500: ---------------------------------------- Dear [~ryannedolan] , thanks also from my side for your efforts pushing MM2. I was trying out MM2 in order to mirror one cluster completely to a downstream one in a distributed connector setup. When running the MM2, I saw at the beginning lot's of ```WorkerSourceTask\{id=mirror-maker-49} Failed to flush, timed out while waiting for producer to flush outstanding 5942 messages``` & ```WorkerSourceTask\{id=mirror-maker-49} Failed to commit offsets``` Error messages. I would guess they are happening when the MM2 replicates faster, than it can write on the connect.offsets.topic. This topic is configured with 25 partitions, which I thought should provide enough throughput to handle the load. In order to test the robustness of the MM2, we also replaced the containers several times during the replication and observed a surprising amount of duplicated messages, which I guess happen because of the lagging offset commit. Any ideas, how the setup can be improved so it becomes more robust. > MirrorMaker 2.0 (KIP-382) > ------------------------- > > Key: KAFKA-7500 > URL: https://issues.apache.org/jira/browse/KAFKA-7500 > Project: Kafka > Issue Type: New Feature > Components: KafkaConnect, mirrormaker > Affects Versions: 2.4.0 > Reporter: Ryanne Dolan > Assignee: Manikumar > Priority: Major > Labels: pull-request-available, ready-to-commit > Fix For: 2.4.0 > > Attachments: Active-Active XDCR setup.png > > > ImplementĀ a drop-in replacement for MirrorMaker leveraging the Connect > framework. > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0] > [https://github.com/apache/kafka/pull/6295] -- This message was sent by Atlassian Jira (v8.3.2#803003)