That's correct. The number of watchers here would be doubled. Also, note that the agent `MasterDetector` is actually checking the leader amongst the `overlay-master` modules and not the Mesos master itself. Since these are anonymous modules there is no way to share the slave/masters objects with the modules (atleast the intention is not to have any sharing). That said, we haven't seen any real impact on Zookeeper due to these extra detectors/watchers being in place in practice. However, I would like to post a disclaimer that this statement is more anecdotal rather than based on some specific experiments. We haven't done any benchmarking ourselves to verify the exact increase in load with these extra detectors in place.
One thing to keep in mind is that Master failovers are relatively rare events (or atleast they should be in a stable system). This implies that while you might see a spike in the load for zookeeper watchers when the event does occur, this is not something that should stress zookeeper out. On Tue, Jan 16, 2018 at 1:15 PM, Zhitao Li <zhitaoli...@gmail.com> wrote: > Hi Avinash, > > Thanks for the pointer. A quick scan seems to suggest that your > implementation requires it to create two different instances of > `MasterDetector`, one for master/agent itself, and one for your module. > That probably means number of watchers on zookeeper would be doubled? Would > that create unnecessary load? > > > > On Tue, Jan 16, 2018 at 11:22 AM, Avinash Sridharan <avin...@mesosphere.io > > > wrote: > > > Hi Zhitao, > > We actually do this in the `DC/OS` overlay modules we use in DC/OS. The > > overlay modules run as Mesos modules both in the Master and the Agent. > You > > can see how we use the `MasterDetector` in the agent module: > > https://github.com/dcos/dcos-mesos-modules/blob/master/ > > overlay/agent.cpp#L74 > > > > https://github.com/dcos/dcos-mesos-modules/blob/master/overlay/ should > > have > > more details of how the modules are actually used to build DC/OS overlay. > > > > On Tue, Jan 16, 2018 at 10:20 AM, Zhitao Li <zhitaoli...@gmail.com> > wrote: > > > > > Hi, > > > > > > Some of our future development work on our custom modules requires them > > to > > > know the current leader of Mesos master. While it seems like we could > > > duplicate the logic in master/slave side to duplicate an instance of > > > `MasterDetector`, it seems more natural if we could figure out a clean > > way > > > to share the existing detector instance in slave to the module. > > > > > > Any comment on this idea? > > > > > > Thanks. > > > > > > -- > > > Cheers, > > > > > > Zhitao Li > > > > > > > > > > > -- > > Avinash Sridharan, Mesosphere > > +1 (323) 702 5245 > > > > > > -- > Cheers, > > Zhitao Li > -- Avinash Sridharan, Mesosphere +1 (323) 702 5245
