Re: Question about controller behavior when fenced

2021-09-29 Thread Neeraj Vaidya
Hi Andrew, Every broker receives a notification about the /controller znode being created and will also receive the latest epoch. The old controller will get some partitions assigned as a result of it being considered as part of the cluster, by the new controller. This is the normal partition a

mm2: size of target vs source topic

2021-09-29 Thread Men Lim
Hi all, I'm setting up two new clusters today and using mm2 to replicate data from clusterA --> clusterB. I noticed that the topic has the same amount of record but the size is small by 5x. source topic is 6.975 MB target topic is: 1.136 MB It has the same number of record. both cluster is usin

Not able to replicate groups in Mirror Maker 2

2021-09-29 Thread Hemanth Savasere
Initially, I created a topic named "quickstart-events" and then produced some messages into it, then consumed it from the kafka-console-consumer with consumer group "quickstartGroup" and now I want to replicate the group from source to destination. When I run describe command to describe the group

MM2 unreachable clusters

2021-09-29 Thread Taylor Mahany
Hello, I have 50 on-prem field devices that each function as a single-broker kafka cluster. I am replicating to a multi-broker "master" cluster in a datacenter. I've been doing this by running the original mirrormaker in the datacenter. (Because of concerns about unauthorized data access, we ca

Question about controller behavior when fenced

2021-09-29 Thread Andrew Grant
Hi all, I had a question about controller behavior when fenced. From my understanding epoch numbers are used to fence brokers who might think they're still the controller but really in the meantime a new broker has been elected as the new controller. My question is, how does a broker realize it's

Not able to replicate groups in Mirror Maker 2

2021-09-29 Thread Hemanth Savasere
Hi, Initially, I created a topic named "quickstart-events" and then produced some messages into it, then consumed it from the kafka-console-consumer with consumer group "quickstartGroup" and now I want to replicate the group from source to destination. When I run describe command to describe the