Hmm, that's weird. I'd expect the type of tasks to be evenly distributed as well. Is it possible one of the internal topics are misconfigured s.t. the Herders aren't functioning correctly?
Ryanne On Fri, Mar 20, 2020 at 11:17 AM Péter Sinóros-Szabó <peter.sinoros-sz...@transferwise.com.invalid> wrote: > I use tasks.max = 4. > > I see 4 tasks of MirrorSourceConnectors on MM2 instances A. > I see 4 tasks of MirrorCheckpointConnector and 1 task of > MirrorHeartbeatConnector on MM2 instance B. > > The number of tasks are well distributed, but the type of tasks are not. > According to Connect documentation I expected 1-3 or 2-2 tasks of > the MirrorSourceConnectors on the two MM2 instances. > > So is this a bug or an expected behaviour? > > Thanks, > Peter > > On Fri, 20 Mar 2020 at 15:26, Ryanne Dolan <ryannedo...@gmail.com> wrote: > > > Peter, in Connect the Connectors are only run on the leader node. Most of > > the work is done in the Tasks, which should be divided across nodes. Make > > sure you have tasks.max set to something higher than the default of 1. > > > > Ryanne > > > > On Fri, Mar 20, 2020, 8:53 AM Péter Sinóros-Szabó > > <peter.sinoros-sz...@transferwise.com.invalid> wrote: > > > > > Hey, > > > > > > I am using MM2 to mirror A cluster to B with tasks.max = 4. > > > > > > I started two instances of MM2 and noticed that all > > MirrorSourceConnectors > > > were running in one instance and the rest of the connectors in the > other. > > > > > > This results in a very uneven resource utilization and also it did not > > > really spread the mirroring oad between the two nodes. > > > > > > I assumed that MM2 will run 2-2 of those connectors in each instance. > > > > > > Is this current behaviour as expected or did I miss something on how to > > > configure it better? > > > > > > Thanks, > > > Peter > > > > > >
