Hi all I'm testing Kafka MM2, and noticed topic and data are not replicated, could anyone help, thanks a lot!
My kafka version is kafka_2.13-2.8.0, two kafka cluster: sc2-dev-shared-kafka01 and sc2-dev-shared-kafka02, MM2 processes runs in each node of kafka clusters(3 MM2 processes in kafka01 and 3 MM2 processes in kafka02). I config MM2 in kafka01 to replicate data to kafk02, I config MM2 in kafka02 to replicate data to kafka01. MM2 config in kafka01: clusters=sc2-dev-shared-kafka01,sc2-dev-shared-kafka02 sc2-dev-shared-kafka01.bootstrap.servers=sc2-dev-shared-kafka01a.eng.vmware.com:9092, sc2-dev-shared-kafka01b.eng.vmware.com:9092, sc2-dev-shared-kafka01c.eng.vmware.com:9092 sc2-dev-shared-kafka02.bootstrap.servers=sc2-dev-shared-kafka02a.eng.vmware.com:9092, sc2-dev-shared-kafka02b.eng.vmware.com:9092, sc2-dev-shared-kafka02c.eng.vmware.com:9092 sc2-dev-shared-kafka01->sc2-dev-shared-kafka02.enabled=true sc2-dev-shared-kafka02->sc2-dev-shared-kafka01.enabled=false topics=.* groups=.* tasks.max=10 checkpoints.topic.replication.factor=2 heartbeats.topic.replication.factor=2 offset-syncs.topic.replication.factor=2 offset.storage.replication.factor=2 status.storage.replication.factor=2 config.storage.replication.factor=2 MM2 config in kafka02: clusters=sc2-dev-shared-kafka01,sc2-dev-shared-kafka02 sc2-dev-shared-kafka01.bootstrap.servers=sc2-dev-shared-kafka01a.eng.vmware.com:9092, sc2-dev-shared-kafka01b.eng.vmware.com:9092, sc2-dev-shared-kafka01c.eng.vmware.com:9092 sc2-dev-shared-kafka02.bootstrap.servers=sc2-dev-shared-kafka02a.eng.vmware.com:9092, sc2-dev-shared-kafka02b.eng.vmware.com:9092, sc2-dev-shared-kafka02c.eng.vmware.com:9092 sc2-dev-shared-kafka01->sc2-dev-shared-kafka02.enabled=false sc2-dev-shared-kafka02->sc2-dev-shared-kafka01.enabled=true topics=.* groups=.* tasks.max=10 checkpoints.topic.replication.factor=2 heartbeats.topic.replication.factor=2 offset-syncs.topic.replication.factor=2 offset.storage.replication.factor=2 status.storage.replication.factor=2 config.storage.replication.factor=2 I make MM2 run as service in each kafka node, and service is good, there is no error from MM2 logs. I create topic-20 and topic-25 in different kafka clusters, I hope those topics will be replicated to remote cluster, but only part of them are replicated, it has been 30 minutes past, I think there should be some problem here, and I see topics list in those two cluster are different, looks like sc2-dev-shared-kafka02 missed some topic which should be replicated from sc2-dev-shared-kafka01. /mnt/hgfs/VmShared/binary/Kafka-Storm/kafka_2.13-2.7.0 > ./bin/kafka-topics.sh --zookeeper sc2-dev-shared-kafka01a:2181 --list __consumer_offsets _schemas heartbeats mm2-configs.sc2-dev-shared-kafka02.internal mm2-offsets.sc2-dev-shared-kafka02.internal mm2-status.sc2-dev-shared-kafka02.internal sc2-dev-shared-kafka02._schemas sc2-dev-shared-kafka02.checkpoints.internal sc2-dev-shared-kafka02.heartbeats sc2-dev-shared-kafka02.topic-25 topic-20 /mnt/hgfs/VmShared/binary/Kafka-Storm/kafka_2.13-2.7.0 > ./bin/kafka-topics.sh --zookeeper sc2-dev-shared-kafka02a:2181 --list __consumer_offsets _schemas heartbeats mm2-configs.sc2-dev-shared-kafka01.internal mm2-offset-syncs.sc2-dev-shared-kafka01.internal mm2-offsets.sc2-dev-shared-kafka01.internal mm2-status.sc2-dev-shared-kafka01.internal topic-25 /mnt/hgfs/VmShared/binary/Kafka-Storm/kafka_2.13-2.7.0 > Also, when I send message to topic-25 in sc2-dev-shared-kafka02, I don't see data is replicated to sc2-dev-shared-kafka01: /mnt/hgfs/VmShared/binary/Kafka-Storm/kafka_2.13-2.7.0 > ./bin/kafka-console-producer.sh --bootstrap-server sc2-dev-shared-kafka02a:9092 --topic topic-25 >hello >do you copy? > /mnt/hgfs/VmShared/binary/Kafka-Storm/kafka_2.13-2.7.0 > ./bin/kafka-console-consumer.sh --bootstrap-server sc2-dev-shared-kafka01a:9092 --topic sc2-dev-shared-kafka02.topic-25 --from-beginning
