MirrorMaker 2 Reload Configuration

2020-10-29 Thread Devaki, Srinivas
Hi Folks, I'm running mirror maker as a dedicated cluster as given in the mirrormaker 2 doc. but for some reason when I add new topics and deploy the mirror maker it's not detecting the new topics at all, even the config dumps in the mirror maker startup logs don't show the newly added topics. I'

Re: MirrorMaker 2 Reload Configuration

2020-10-29 Thread Devaki, Srinivas
Hi Folks, I've also ran a console-consumer on the `mm2-configs` kafka topic created by the mirror maker and found that even after the restart of the mirror maker 2 with new config, the config registered in the mm2-configs kafka topic is still pointing to a legacy mirror maker configuration. Thank

Re: MirrorMaker 2 Reload Configuration

2020-11-04 Thread Ning Zhang
if your new topics are not named "topic1" or "topic2", maybe you want to use regex * to allow more topics to be considered by Mm2 # regex which defines which topics gets replicated. For eg "foo-.*" src-cluster->dst-cluster.topics = topic1,topic2 On 2020/10/30 01:48:00, "Devaki, Srinivas" wrote:

Re: MirrorMaker 2 Reload Configuration

2020-11-11 Thread Péter Sinóros-Szabó
Hi, I have a similar issue. I changed the source cluster bootstrap address and MM2 picked it up only partially. Some parts of it still use the old address, some the new. The old and the new address list is routed to the same cluster, same brokers, just on a different network path. So is there an

Re: MirrorMaker 2 Reload Configuration

2020-11-11 Thread Ryanne Dolan
Hey guys, this is because the configuration gets loaded into the internal mm2-config topics, and these may get out of sync with the mm2.properties file in some scenarios. I believe this occurs whenever an old/bad configuration gets written to Kafka, which MM2 can read successfully but which causes

Re: MirrorMaker 2 Reload Configuration

2020-11-13 Thread Péter Sinóros-Szabó
Hi Ryanne, I will open an issue in Jira. I see mm2-config and mm2-status topics on both the source and destination clusters. Should I purge all of them? Or is it enough to purge just the destination topics? Thanks, Peter On Wed, 11 Nov 2020 at 19:33, Ryanne Dolan wrote: > Hey guys, this is bec

Re: MirrorMaker 2 Reload Configuration

2020-11-13 Thread Devaki, Srinivas
Hi All, After inspecting a few internal topics and running console consumer to see the payloads in the mm2-configs topic identified that properties are indeed not getting refreshed, I've assumed that mm2 internally is joining the existing cluster, so to refresh config I've tried to completely stop

Re: MirrorMaker 2 Reload Configuration

2020-11-13 Thread Péter Sinóros-Szabó
Hi, I tried as well to stop all instances of MM2, but it didn't help for me. I had to stop all MM2 instances, delete the mm2-config and mm2-status topics on the destination cluster and start up all MM2 instances again. Peter

RE: Re: MirrorMaker 2 Reload Configuration

2022-01-13 Thread Praveen Sinha
Hi Peter, I am running into a similar issue, did you create a JIRA ticket for this? Or is there any workaround you have found for the same? Thanks and regards, Praveen On 2020/11/13 14:45:23 Péter Sinóros-Szabó wrote: > Hi, > > I tried as well to stop all instances of MM2, but it didn't help for