Re: MirrorMaker2 - uneven loadbalancing
I bumped tasks.max to 10, but it does not help if I only have two instances: all MirrorSourceConnector tasks are still in one instance. But if I start more instances in case I need more bandwidth, it will help to spread the load, so this is ok for me so far. Also KAFKA-9352 will help to get better instance usage, once Connect assigns tasks in all instances :) Peter On Mon, 23 Mar 2020 at 19:07, Ryanne Dolan wrote: > Thanks Peter for running this experiment. That looks sorta normal. It looks > like Connect is deciding to use 10 total tasks and doesn't care which ones > do what. Ideally you'd see the MirrorSourceConnector tasks evenly divided, > since they do the bulk of the work -- but that doesn't seem to be the case > with your selection of parameters. > > I'd recommend bumping up the tasks.max a lot higher than 4 in order to > achieve finer-grained workloads and a more even balance. > > Ryanne > > On Mon, Mar 23, 2020 at 9:58 AM Péter Sinóros-Szabó > wrote: > > > so I made some tests with tasks.max = 4 > > > > with 2 instances: > > - instance 1: 4 MirrorSourceConnector, 1 MirrorHeartbeatConnector tasks > > - instance 2: 4 MirrorCheckpointConnector, 1 MirrorHeartbeatConnector > tasks > > > > with 3 instances: > > - instance 1: 3 MirrorCheckpointConnector tasks > > - instance 2: 3 MirrorSourceConnector tasks, 1 MirrorHeartbeatConnector > > task > > - instance 3: 1 MirrorSourceConnector, 1 MirrorCheckpointConnector task, > > 1 MirrorHeartbeatConnector task > > > > with 4 instances: > > - instance 1: 3 MirrorCheckpointConnector tasks > > - instance 2: 2 MirrorSourceConnector tasks, 1 MirrorHeartbeatConnector > > task > > - instance 3: 1 MirrorSourceConnector task, 1 MirrorCheckpointConnector > > task > > - instance 4: 1 MirrorSourceConnector task, 1 MirrorHeartbeatConnector > task > > > > So it seems that it is not well balanced, but can be scaled somewhat, not > > ideal. > > Is this how it should work? > > > > Peter > > > > On Fri, 20 Mar 2020 at 20:58, Ryanne Dolan > wrote: > > > > > Peter, what happens when you add an additional node? Usually Connect > will > > > detect it and rebalance tasks accordingly. I'm wondering if that > > mechanism > > > isn't working for you. > > > > > > Ryanne > > > > > > On Fri, Mar 20, 2020 at 2:40 PM Péter Sinóros-Szabó > > > wrote: > > > > > > > Well, I don't know much about herders. If you can give some idea how > to > > > > check it, I will try. > > > > > > > > Peter > > > > > > > > On Fri, 20 Mar 2020 at 17:47, Ryanne Dolan > > > wrote: > > > > > > > > > Hmm, that's weird. I'd expect the type of tasks to be evenly > > > distributed > > > > as > > > > > well. Is it possible one of the internal topics are misconfigured > > s.t. > > > > the > > > > > Herders aren't functioning correctly? > > > > > > > > > > Ryanne > > > > > > > > > > On Fri, Mar 20, 2020 at 11:17 AM Péter Sinóros-Szabó > > > > > wrote: > > > > > > > > > > > I use tasks.max = 4. > > > > > > > > > > > > I see 4 tasks of MirrorSourceConnectors on MM2 instances A. > > > > > > I see 4 tasks of MirrorCheckpointConnector and 1 task of > > > > > > MirrorHeartbeatConnector on MM2 instance B. > > > > > > > > > > > > The number of tasks are well distributed, but the type of tasks > are > > > > not. > > > > > > According to Connect documentation I expected 1-3 or 2-2 tasks of > > > > > > the MirrorSourceConnectors on the two MM2 instances. > > > > > > > > > > > > So is this a bug or an expected behaviour? > > > > > > > > > > > > Thanks, > > > > > > Peter > > > > > > > > > > > > On Fri, 20 Mar 2020 at 15:26, Ryanne Dolan < > ryannedo...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > Peter, in Connect the Connectors are only run on the leader > node. > > > > Most > > > > > of > > > > > > > the work is done in the Tasks, which should be divided across > > > nodes. > > > > > Make > > > > > > > sure you have tasks.max set to something higher than the > default > > of > > > > 1. > > > > > > > > > > > > > > Ryanne > > > > > > > > > > > > > > On Fri, Mar 20, 2020, 8:53 AM Péter Sinóros-Szabó > > > > > > > wrote: > > > > > > > > > > > > > > > Hey, > > > > > > > > > > > > > > > > I am using MM2 to mirror A cluster to B with tasks.max = 4. > > > > > > > > > > > > > > > > I started two instances of MM2 and noticed that all > > > > > > > MirrorSourceConnectors > > > > > > > > were running in one instance and the rest of the connectors > in > > > the > > > > > > other. > > > > > > > > > > > > > > > > This results in a very uneven resource utilization and also > it > > > did > > > > > not > > > > > > > > really spread the mirroring oad between the two nodes. > > > > > > > > > > > > > > > > I assumed that MM2 will run 2-2 of those connectors in each > > > > instance. > > > > > > > > > > > > > > > > Is this current behaviour as expected or did I miss something > > on > > > > how > > > > > to > > > > > > > > configure it better? > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Peter > > > > > > > > > > > > > > >
Re: MirrorMaker2 - uneven loadbalancing
Hi Peter, Not sure this is what you're looking for - https://issues.apache.org/jira/browse/KAFKA-9352 On Mon, Mar 23, 2020 at 11:37 PM Ryanne Dolan wrote: > Thanks Peter for running this experiment. That looks sorta normal. It looks > like Connect is deciding to use 10 total tasks and doesn't care which ones > do what. Ideally you'd see the MirrorSourceConnector tasks evenly divided, > since they do the bulk of the work -- but that doesn't seem to be the case > with your selection of parameters. > > I'd recommend bumping up the tasks.max a lot higher than 4 in order to > achieve finer-grained workloads and a more even balance. > > Ryanne > > On Mon, Mar 23, 2020 at 9:58 AM Péter Sinóros-Szabó > wrote: > > > so I made some tests with tasks.max = 4 > > > > with 2 instances: > > - instance 1: 4 MirrorSourceConnector, 1 MirrorHeartbeatConnector tasks > > - instance 2: 4 MirrorCheckpointConnector, 1 MirrorHeartbeatConnector > tasks > > > > with 3 instances: > > - instance 1: 3 MirrorCheckpointConnector tasks > > - instance 2: 3 MirrorSourceConnector tasks, 1 MirrorHeartbeatConnector > > task > > - instance 3: 1 MirrorSourceConnector, 1 MirrorCheckpointConnector task, > > 1 MirrorHeartbeatConnector task > > > > with 4 instances: > > - instance 1: 3 MirrorCheckpointConnector tasks > > - instance 2: 2 MirrorSourceConnector tasks, 1 MirrorHeartbeatConnector > > task > > - instance 3: 1 MirrorSourceConnector task, 1 MirrorCheckpointConnector > > task > > - instance 4: 1 MirrorSourceConnector task, 1 MirrorHeartbeatConnector > task > > > > So it seems that it is not well balanced, but can be scaled somewhat, not > > ideal. > > Is this how it should work? > > > > Peter > > > > On Fri, 20 Mar 2020 at 20:58, Ryanne Dolan > wrote: > > > > > Peter, what happens when you add an additional node? Usually Connect > will > > > detect it and rebalance tasks accordingly. I'm wondering if that > > mechanism > > > isn't working for you. > > > > > > Ryanne > > > > > > On Fri, Mar 20, 2020 at 2:40 PM Péter Sinóros-Szabó > > > wrote: > > > > > > > Well, I don't know much about herders. If you can give some idea how > to > > > > check it, I will try. > > > > > > > > Peter > > > > > > > > On Fri, 20 Mar 2020 at 17:47, Ryanne Dolan > > > wrote: > > > > > > > > > Hmm, that's weird. I'd expect the type of tasks to be evenly > > > distributed > > > > as > > > > > well. Is it possible one of the internal topics are misconfigured > > s.t. > > > > the > > > > > Herders aren't functioning correctly? > > > > > > > > > > Ryanne > > > > > > > > > > On Fri, Mar 20, 2020 at 11:17 AM Péter Sinóros-Szabó > > > > > wrote: > > > > > > > > > > > I use tasks.max = 4. > > > > > > > > > > > > I see 4 tasks of MirrorSourceConnectors on MM2 instances A. > > > > > > I see 4 tasks of MirrorCheckpointConnector and 1 task of > > > > > > MirrorHeartbeatConnector on MM2 instance B. > > > > > > > > > > > > The number of tasks are well distributed, but the type of tasks > are > > > > not. > > > > > > According to Connect documentation I expected 1-3 or 2-2 tasks of > > > > > > the MirrorSourceConnectors on the two MM2 instances. > > > > > > > > > > > > So is this a bug or an expected behaviour? > > > > > > > > > > > > Thanks, > > > > > > Peter > > > > > > > > > > > > On Fri, 20 Mar 2020 at 15:26, Ryanne Dolan < > ryannedo...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > Peter, in Connect the Connectors are only run on the leader > node. > > > > Most > > > > > of > > > > > > > the work is done in the Tasks, which should be divided across > > > nodes. > > > > > Make > > > > > > > sure you have tasks.max set to something higher than the > default > > of > > > > 1. > > > > > > > > > > > > > > Ryanne > > > > > > > > > > > > > > On Fri, Mar 20, 2020, 8:53 AM Péter Sinóros-Szabó > > > > > > > wrote: > > > > > > > > > > > > > > > Hey, > > > > > > > > > > > > > > > > I am using MM2 to mirror A cluster to B with tasks.max = 4. > > > > > > > > > > > > > > > > I started two instances of MM2 and noticed that all > > > > > > > MirrorSourceConnectors > > > > > > > > were running in one instance and the rest of the connectors > in > > > the > > > > > > other. > > > > > > > > > > > > > > > > This results in a very uneven resource utilization and also > it > > > did > > > > > not > > > > > > > > really spread the mirroring oad between the two nodes. > > > > > > > > > > > > > > > > I assumed that MM2 will run 2-2 of those connectors in each > > > > instance. > > > > > > > > > > > > > > > > Is this current behaviour as expected or did I miss something > > on > > > > how > > > > > to > > > > > > > > configure it better? > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Peter > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: MirrorMaker2 - uneven loadbalancing
Thanks Peter for running this experiment. That looks sorta normal. It looks like Connect is deciding to use 10 total tasks and doesn't care which ones do what. Ideally you'd see the MirrorSourceConnector tasks evenly divided, since they do the bulk of the work -- but that doesn't seem to be the case with your selection of parameters. I'd recommend bumping up the tasks.max a lot higher than 4 in order to achieve finer-grained workloads and a more even balance. Ryanne On Mon, Mar 23, 2020 at 9:58 AM Péter Sinóros-Szabó wrote: > so I made some tests with tasks.max = 4 > > with 2 instances: > - instance 1: 4 MirrorSourceConnector, 1 MirrorHeartbeatConnector tasks > - instance 2: 4 MirrorCheckpointConnector, 1 MirrorHeartbeatConnector tasks > > with 3 instances: > - instance 1: 3 MirrorCheckpointConnector tasks > - instance 2: 3 MirrorSourceConnector tasks, 1 MirrorHeartbeatConnector > task > - instance 3: 1 MirrorSourceConnector, 1 MirrorCheckpointConnector task, > 1 MirrorHeartbeatConnector task > > with 4 instances: > - instance 1: 3 MirrorCheckpointConnector tasks > - instance 2: 2 MirrorSourceConnector tasks, 1 MirrorHeartbeatConnector > task > - instance 3: 1 MirrorSourceConnector task, 1 MirrorCheckpointConnector > task > - instance 4: 1 MirrorSourceConnector task, 1 MirrorHeartbeatConnector task > > So it seems that it is not well balanced, but can be scaled somewhat, not > ideal. > Is this how it should work? > > Peter > > On Fri, 20 Mar 2020 at 20:58, Ryanne Dolan wrote: > > > Peter, what happens when you add an additional node? Usually Connect will > > detect it and rebalance tasks accordingly. I'm wondering if that > mechanism > > isn't working for you. > > > > Ryanne > > > > On Fri, Mar 20, 2020 at 2:40 PM Péter Sinóros-Szabó > > wrote: > > > > > Well, I don't know much about herders. If you can give some idea how to > > > check it, I will try. > > > > > > Peter > > > > > > On Fri, 20 Mar 2020 at 17:47, Ryanne Dolan > > wrote: > > > > > > > Hmm, that's weird. I'd expect the type of tasks to be evenly > > distributed > > > as > > > > well. Is it possible one of the internal topics are misconfigured > s.t. > > > the > > > > Herders aren't functioning correctly? > > > > > > > > Ryanne > > > > > > > > On Fri, Mar 20, 2020 at 11:17 AM Péter Sinóros-Szabó > > > > wrote: > > > > > > > > > I use tasks.max = 4. > > > > > > > > > > I see 4 tasks of MirrorSourceConnectors on MM2 instances A. > > > > > I see 4 tasks of MirrorCheckpointConnector and 1 task of > > > > > MirrorHeartbeatConnector on MM2 instance B. > > > > > > > > > > The number of tasks are well distributed, but the type of tasks are > > > not. > > > > > According to Connect documentation I expected 1-3 or 2-2 tasks of > > > > > the MirrorSourceConnectors on the two MM2 instances. > > > > > > > > > > So is this a bug or an expected behaviour? > > > > > > > > > > Thanks, > > > > > Peter > > > > > > > > > > On Fri, 20 Mar 2020 at 15:26, Ryanne Dolan > > > > wrote: > > > > > > > > > > > Peter, in Connect the Connectors are only run on the leader node. > > > Most > > > > of > > > > > > the work is done in the Tasks, which should be divided across > > nodes. > > > > Make > > > > > > sure you have tasks.max set to something higher than the default > of > > > 1. > > > > > > > > > > > > Ryanne > > > > > > > > > > > > On Fri, Mar 20, 2020, 8:53 AM Péter Sinóros-Szabó > > > > > > wrote: > > > > > > > > > > > > > Hey, > > > > > > > > > > > > > > I am using MM2 to mirror A cluster to B with tasks.max = 4. > > > > > > > > > > > > > > I started two instances of MM2 and noticed that all > > > > > > MirrorSourceConnectors > > > > > > > were running in one instance and the rest of the connectors in > > the > > > > > other. > > > > > > > > > > > > > > This results in a very uneven resource utilization and also it > > did > > > > not > > > > > > > really spread the mirroring oad between the two nodes. > > > > > > > > > > > > > > I assumed that MM2 will run 2-2 of those connectors in each > > > instance. > > > > > > > > > > > > > > Is this current behaviour as expected or did I miss something > on > > > how > > > > to > > > > > > > configure it better? > > > > > > > > > > > > > > Thanks, > > > > > > > Peter > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: MirrorMaker2 - uneven loadbalancing
so I made some tests with tasks.max = 4 with 2 instances: - instance 1: 4 MirrorSourceConnector, 1 MirrorHeartbeatConnector tasks - instance 2: 4 MirrorCheckpointConnector, 1 MirrorHeartbeatConnector tasks with 3 instances: - instance 1: 3 MirrorCheckpointConnector tasks - instance 2: 3 MirrorSourceConnector tasks, 1 MirrorHeartbeatConnector task - instance 3: 1 MirrorSourceConnector, 1 MirrorCheckpointConnector task, 1 MirrorHeartbeatConnector task with 4 instances: - instance 1: 3 MirrorCheckpointConnector tasks - instance 2: 2 MirrorSourceConnector tasks, 1 MirrorHeartbeatConnector task - instance 3: 1 MirrorSourceConnector task, 1 MirrorCheckpointConnector task - instance 4: 1 MirrorSourceConnector task, 1 MirrorHeartbeatConnector task So it seems that it is not well balanced, but can be scaled somewhat, not ideal. Is this how it should work? Peter On Fri, 20 Mar 2020 at 20:58, Ryanne Dolan wrote: > Peter, what happens when you add an additional node? Usually Connect will > detect it and rebalance tasks accordingly. I'm wondering if that mechanism > isn't working for you. > > Ryanne > > On Fri, Mar 20, 2020 at 2:40 PM Péter Sinóros-Szabó > wrote: > > > Well, I don't know much about herders. If you can give some idea how to > > check it, I will try. > > > > Peter > > > > On Fri, 20 Mar 2020 at 17:47, Ryanne Dolan > wrote: > > > > > Hmm, that's weird. I'd expect the type of tasks to be evenly > distributed > > as > > > well. Is it possible one of the internal topics are misconfigured s.t. > > the > > > Herders aren't functioning correctly? > > > > > > Ryanne > > > > > > On Fri, Mar 20, 2020 at 11:17 AM Péter Sinóros-Szabó > > > wrote: > > > > > > > I use tasks.max = 4. > > > > > > > > I see 4 tasks of MirrorSourceConnectors on MM2 instances A. > > > > I see 4 tasks of MirrorCheckpointConnector and 1 task of > > > > MirrorHeartbeatConnector on MM2 instance B. > > > > > > > > The number of tasks are well distributed, but the type of tasks are > > not. > > > > According to Connect documentation I expected 1-3 or 2-2 tasks of > > > > the MirrorSourceConnectors on the two MM2 instances. > > > > > > > > So is this a bug or an expected behaviour? > > > > > > > > Thanks, > > > > Peter > > > > > > > > On Fri, 20 Mar 2020 at 15:26, Ryanne Dolan > > > wrote: > > > > > > > > > Peter, in Connect the Connectors are only run on the leader node. > > Most > > > of > > > > > the work is done in the Tasks, which should be divided across > nodes. > > > Make > > > > > sure you have tasks.max set to something higher than the default of > > 1. > > > > > > > > > > Ryanne > > > > > > > > > > On Fri, Mar 20, 2020, 8:53 AM Péter Sinóros-Szabó > > > > > wrote: > > > > > > > > > > > Hey, > > > > > > > > > > > > I am using MM2 to mirror A cluster to B with tasks.max = 4. > > > > > > > > > > > > I started two instances of MM2 and noticed that all > > > > > MirrorSourceConnectors > > > > > > were running in one instance and the rest of the connectors in > the > > > > other. > > > > > > > > > > > > This results in a very uneven resource utilization and also it > did > > > not > > > > > > really spread the mirroring oad between the two nodes. > > > > > > > > > > > > I assumed that MM2 will run 2-2 of those connectors in each > > instance. > > > > > > > > > > > > Is this current behaviour as expected or did I miss something on > > how > > > to > > > > > > configure it better? > > > > > > > > > > > > Thanks, > > > > > > Peter > > > > > > > > > > > > > > > > > > > > >
Re: MirrorMaker2 - uneven loadbalancing
Peter, what happens when you add an additional node? Usually Connect will detect it and rebalance tasks accordingly. I'm wondering if that mechanism isn't working for you. Ryanne On Fri, Mar 20, 2020 at 2:40 PM Péter Sinóros-Szabó wrote: > Well, I don't know much about herders. If you can give some idea how to > check it, I will try. > > Peter > > On Fri, 20 Mar 2020 at 17:47, Ryanne Dolan wrote: > > > Hmm, that's weird. I'd expect the type of tasks to be evenly distributed > as > > well. Is it possible one of the internal topics are misconfigured s.t. > the > > Herders aren't functioning correctly? > > > > Ryanne > > > > On Fri, Mar 20, 2020 at 11:17 AM Péter Sinóros-Szabó > > wrote: > > > > > I use tasks.max = 4. > > > > > > I see 4 tasks of MirrorSourceConnectors on MM2 instances A. > > > I see 4 tasks of MirrorCheckpointConnector and 1 task of > > > MirrorHeartbeatConnector on MM2 instance B. > > > > > > The number of tasks are well distributed, but the type of tasks are > not. > > > According to Connect documentation I expected 1-3 or 2-2 tasks of > > > the MirrorSourceConnectors on the two MM2 instances. > > > > > > So is this a bug or an expected behaviour? > > > > > > Thanks, > > > Peter > > > > > > On Fri, 20 Mar 2020 at 15:26, Ryanne Dolan > > wrote: > > > > > > > Peter, in Connect the Connectors are only run on the leader node. > Most > > of > > > > the work is done in the Tasks, which should be divided across nodes. > > Make > > > > sure you have tasks.max set to something higher than the default of > 1. > > > > > > > > Ryanne > > > > > > > > On Fri, Mar 20, 2020, 8:53 AM Péter Sinóros-Szabó > > > > wrote: > > > > > > > > > Hey, > > > > > > > > > > I am using MM2 to mirror A cluster to B with tasks.max = 4. > > > > > > > > > > I started two instances of MM2 and noticed that all > > > > MirrorSourceConnectors > > > > > were running in one instance and the rest of the connectors in the > > > other. > > > > > > > > > > This results in a very uneven resource utilization and also it did > > not > > > > > really spread the mirroring oad between the two nodes. > > > > > > > > > > I assumed that MM2 will run 2-2 of those connectors in each > instance. > > > > > > > > > > Is this current behaviour as expected or did I miss something on > how > > to > > > > > configure it better? > > > > > > > > > > Thanks, > > > > > Peter > > > > > > > > > > > > > > >
Re: MirrorMaker2 - uneven loadbalancing
Well, I don't know much about herders. If you can give some idea how to check it, I will try. Peter On Fri, 20 Mar 2020 at 17:47, Ryanne Dolan wrote: > Hmm, that's weird. I'd expect the type of tasks to be evenly distributed as > well. Is it possible one of the internal topics are misconfigured s.t. the > Herders aren't functioning correctly? > > Ryanne > > On Fri, Mar 20, 2020 at 11:17 AM Péter Sinóros-Szabó > wrote: > > > I use tasks.max = 4. > > > > I see 4 tasks of MirrorSourceConnectors on MM2 instances A. > > I see 4 tasks of MirrorCheckpointConnector and 1 task of > > MirrorHeartbeatConnector on MM2 instance B. > > > > The number of tasks are well distributed, but the type of tasks are not. > > According to Connect documentation I expected 1-3 or 2-2 tasks of > > the MirrorSourceConnectors on the two MM2 instances. > > > > So is this a bug or an expected behaviour? > > > > Thanks, > > Peter > > > > On Fri, 20 Mar 2020 at 15:26, Ryanne Dolan > wrote: > > > > > Peter, in Connect the Connectors are only run on the leader node. Most > of > > > the work is done in the Tasks, which should be divided across nodes. > Make > > > sure you have tasks.max set to something higher than the default of 1. > > > > > > Ryanne > > > > > > On Fri, Mar 20, 2020, 8:53 AM Péter Sinóros-Szabó > > > wrote: > > > > > > > Hey, > > > > > > > > I am using MM2 to mirror A cluster to B with tasks.max = 4. > > > > > > > > I started two instances of MM2 and noticed that all > > > MirrorSourceConnectors > > > > were running in one instance and the rest of the connectors in the > > other. > > > > > > > > This results in a very uneven resource utilization and also it did > not > > > > really spread the mirroring oad between the two nodes. > > > > > > > > I assumed that MM2 will run 2-2 of those connectors in each instance. > > > > > > > > Is this current behaviour as expected or did I miss something on how > to > > > > configure it better? > > > > > > > > Thanks, > > > > Peter > > > > > > > > > >
Re: MirrorMaker2 - uneven loadbalancing
Hmm, that's weird. I'd expect the type of tasks to be evenly distributed as well. Is it possible one of the internal topics are misconfigured s.t. the Herders aren't functioning correctly? Ryanne On Fri, Mar 20, 2020 at 11:17 AM Péter Sinóros-Szabó wrote: > I use tasks.max = 4. > > I see 4 tasks of MirrorSourceConnectors on MM2 instances A. > I see 4 tasks of MirrorCheckpointConnector and 1 task of > MirrorHeartbeatConnector on MM2 instance B. > > The number of tasks are well distributed, but the type of tasks are not. > According to Connect documentation I expected 1-3 or 2-2 tasks of > the MirrorSourceConnectors on the two MM2 instances. > > So is this a bug or an expected behaviour? > > Thanks, > Peter > > On Fri, 20 Mar 2020 at 15:26, Ryanne Dolan wrote: > > > Peter, in Connect the Connectors are only run on the leader node. Most of > > the work is done in the Tasks, which should be divided across nodes. Make > > sure you have tasks.max set to something higher than the default of 1. > > > > Ryanne > > > > On Fri, Mar 20, 2020, 8:53 AM Péter Sinóros-Szabó > > wrote: > > > > > Hey, > > > > > > I am using MM2 to mirror A cluster to B with tasks.max = 4. > > > > > > I started two instances of MM2 and noticed that all > > MirrorSourceConnectors > > > were running in one instance and the rest of the connectors in the > other. > > > > > > This results in a very uneven resource utilization and also it did not > > > really spread the mirroring oad between the two nodes. > > > > > > I assumed that MM2 will run 2-2 of those connectors in each instance. > > > > > > Is this current behaviour as expected or did I miss something on how to > > > configure it better? > > > > > > Thanks, > > > Peter > > > > > >
Re: MirrorMaker2 - uneven loadbalancing
I use tasks.max = 4. I see 4 tasks of MirrorSourceConnectors on MM2 instances A. I see 4 tasks of MirrorCheckpointConnector and 1 task of MirrorHeartbeatConnector on MM2 instance B. The number of tasks are well distributed, but the type of tasks are not. According to Connect documentation I expected 1-3 or 2-2 tasks of the MirrorSourceConnectors on the two MM2 instances. So is this a bug or an expected behaviour? Thanks, Peter On Fri, 20 Mar 2020 at 15:26, Ryanne Dolan wrote: > Peter, in Connect the Connectors are only run on the leader node. Most of > the work is done in the Tasks, which should be divided across nodes. Make > sure you have tasks.max set to something higher than the default of 1. > > Ryanne > > On Fri, Mar 20, 2020, 8:53 AM Péter Sinóros-Szabó > wrote: > > > Hey, > > > > I am using MM2 to mirror A cluster to B with tasks.max = 4. > > > > I started two instances of MM2 and noticed that all > MirrorSourceConnectors > > were running in one instance and the rest of the connectors in the other. > > > > This results in a very uneven resource utilization and also it did not > > really spread the mirroring oad between the two nodes. > > > > I assumed that MM2 will run 2-2 of those connectors in each instance. > > > > Is this current behaviour as expected or did I miss something on how to > > configure it better? > > > > Thanks, > > Peter > > >
Re: MirrorMaker2 - uneven loadbalancing
Peter, in Connect the Connectors are only run on the leader node. Most of the work is done in the Tasks, which should be divided across nodes. Make sure you have tasks.max set to something higher than the default of 1. Ryanne On Fri, Mar 20, 2020, 8:53 AM Péter Sinóros-Szabó wrote: > Hey, > > I am using MM2 to mirror A cluster to B with tasks.max = 4. > > I started two instances of MM2 and noticed that all MirrorSourceConnectors > were running in one instance and the rest of the connectors in the other. > > This results in a very uneven resource utilization and also it did not > really spread the mirroring oad between the two nodes. > > I assumed that MM2 will run 2-2 of those connectors in each instance. > > Is this current behaviour as expected or did I miss something on how to > configure it better? > > Thanks, > Peter >
MirrorMaker2 - uneven loadbalancing
Hey, I am using MM2 to mirror A cluster to B with tasks.max = 4. I started two instances of MM2 and noticed that all MirrorSourceConnectors were running in one instance and the rest of the connectors in the other. This results in a very uneven resource utilization and also it did not really spread the mirroring oad between the two nodes. I assumed that MM2 will run 2-2 of those connectors in each instance. Is this current behaviour as expected or did I miss something on how to configure it better? Thanks, Peter
