Have you tried running mm2 in a distributed mode instead of standalone/dedicated? Try using this file below that I have working on a AWS EC2 instance:
name=mm2 clusters = source, target source.bootstrap.servers = source:9092 target.bootstrap.servers = target:9092 group.id = mm2 ## all replication factor has to be greater than the min.insync.replicas = 2, which is the default msk configuration. source.config.storage.replication.factor = 4 target.config.storage.replication.factor = 4 source.offset.storage.replication.factor = 4 target.offset.storage.replication.factor = 4 source.status.storage.replication.factor = 4 target.status.storage.replication.factor = 4 source->target.enabled = true target->source.enabled = false offset-syncs.topic.replication.factor = 4 heartbeats.topic.replication.factor = 4 checkpoints.topic.replication.factor = 4 topics = .* tasks.max = 2 replication.factor = 4 refresh.topics.enabled = true sync.topic.configs.enabled = true refresh.topics.interval.seconds = 30 topics.blacklist = .*[\-\.]internal, .*\.replica, __consumer_offsets groups.blacklist = console-consumer-.*, connect-.*, __.* source->target.emit.heartbeats.enabled=true source->target.emit.checkpoints.enabled=true On Tue, Aug 10, 2021 at 7:54 PM AlR <alp...@gmail.com> wrote: > I have 2 Kafka setup, A and B. A is a cluster with 3 instances running on > the same machine, while B is a stand alone in another machine. I tried to > use Mirror Maker to replicate from A to B. The config file is as follows: > > ----- > > clusters = A, BA.bootstrap.servers = host1:9091, host1:9092, > host1:9093B.bootstrap.servers = host2:9092 > A->B.enabled = trueB->A.enabled = false > replication.factor = 1 > checkpoints.topic.replication.factor=1heartbeats.topic > replication.factor=1offset-syncs.topic.replication.factor=1 > > offset.storage.replication.factor=1status.storage.replication.factor=1config.storage.replication.factor=1--- > > I execute mirror maker as follows: > > bin/connect-mirror-maker.sh config/mirror.maker.properties > > But when I run the mirror maker on the ‘B’ side I can see that it keeps > creating ‘heartbeats’ topic. So at some point I will end up having so many > ‘A.heartbeats’ topics like below. And it will just keep growing until I > kill the mirror maker. > > A.A.A.A.heartbeatsA.A.A.heartbeatsA.A.heartbeatsA.heartbeats > > Anyone know what’s the problem and how to fix this? > > Thanks. > > Kafka Version: 2.13-2.8 >
