Hi Dmitri, There's at least one issue with MirrorMaker 2 that impacts horizontal scalability and has not yet been addressed: https://issues.apache.org/jira/browse/KAFKA-9981. There is some work in progress to fix it ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-710%3A+Full+support+for+distributed+mode+in+dedicated+MirrorMaker+2.0+clusters), but the effort hasn't received much attention to date.
There may be other issues as well, but until KAFKA-9981 is resolved, running MirrorMaker 2 in a multi-node cluster will be at best difficult and at worst, impossible. Cheers, Chris On Thu, Mar 3, 2022 at 10:00 AM Dmitri Pavlov <dpav...@perforce.com> wrote: > Hi, > > A quick question, maybe you can help? > > Trying to follow this article > https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0 > -> "Walkthrough: Running MirrorMaker 2.0", and the last lines in the > paragraph are > > ====== > Second, launch one or more MirrorMaker cluster nodes: > $ ./bin/connect-mirror-maker.sh mm2.properties > ====== > > But apparently it does not work this way, one of 2 simultaneously started > instance will remain idle, confirmed with Jconsole -> Mbeans. > Setup: Broker A (in cluster A) -> MM2 2 instances -> Broker B (cluster B), > for simplicity there is only one broker per cluster. > A simple experiment is, when one instance is configured and started to > replicate topic A only and another topic B only, only one topic will be > replicated, when 2 instances are running in parallel. While, if only one of > the instances is running at a time, each topic will be replicated correctly. > > The main question -> is Mirrormaker 2 horizontally scalable? And if yes, > would be possible to share a link to a document that describes the setup > process? > > Thanks in advance, > Dmitri. > > > This e-mail may contain information that is privileged or confidential. If > you are not the intended recipient, please delete the e-mail and any > attachments and notify us immediately. > >
