If the module needs to detect the same master as the agent does, I think this feature can be implemented as a module hook.
On Tue, Jan 16, 2018 at 1:48 PM, Benjamin Mahler <bmah...@apache.org> wrote: > Just a note that if you increase the number of instances of detectors you > wind up increasing the number of instances of our `Zookeeper` client class. > This class was written a long time ago and so is still a blocking interface > rather than a Future-based non-blocking interface. As a result, the minimum > number of threads needed for libprocess to avoid deadlocking will increase: > > https://issues.apache.org/jira/browse/MESOS-8255 > > On Tue, Jan 16, 2018 at 1:32 PM, Avinash Sridharan <avin...@mesosphere.io> > wrote: > > > That's correct. The number of watchers here would be doubled. Also, note > > that the agent `MasterDetector` is actually checking the leader amongst > the > > `overlay-master` modules and not the Mesos master itself. Since these are > > anonymous modules there is no way to share the slave/masters objects with > > the modules (atleast the intention is not to have any sharing). That > said, > > we haven't seen any real impact on Zookeeper due to these extra > > detectors/watchers being in place in practice. However, I would like to > > post a disclaimer that this statement is more anecdotal rather than based > > on some specific experiments. We haven't done any benchmarking ourselves > to > > verify the exact increase in load with these extra detectors in place. > > > > One thing to keep in mind is that Master failovers are relatively rare > > events (or atleast they should be in a stable system). This implies that > > while you might see a spike in the load for zookeeper watchers when the > > event does occur, this is not something that should stress zookeeper out. > > > > On Tue, Jan 16, 2018 at 1:15 PM, Zhitao Li <zhitaoli...@gmail.com> > wrote: > > > > > Hi Avinash, > > > > > > Thanks for the pointer. A quick scan seems to suggest that your > > > implementation requires it to create two different instances of > > > `MasterDetector`, one for master/agent itself, and one for your module. > > > That probably means number of watchers on zookeeper would be doubled? > > Would > > > that create unnecessary load? > > > > > > > > > > > > On Tue, Jan 16, 2018 at 11:22 AM, Avinash Sridharan < > > avin...@mesosphere.io > > > > > > > wrote: > > > > > > > Hi Zhitao, > > > > We actually do this in the `DC/OS` overlay modules we use in DC/OS. > > The > > > > overlay modules run as Mesos modules both in the Master and the > Agent. > > > You > > > > can see how we use the `MasterDetector` in the agent module: > > > > https://github.com/dcos/dcos-mesos-modules/blob/master/ > > > > overlay/agent.cpp#L74 > > > > > > > > https://github.com/dcos/dcos-mesos-modules/blob/master/overlay/ > should > > > > have > > > > more details of how the modules are actually used to build DC/OS > > overlay. > > > > > > > > On Tue, Jan 16, 2018 at 10:20 AM, Zhitao Li <zhitaoli...@gmail.com> > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > Some of our future development work on our custom modules requires > > them > > > > to > > > > > know the current leader of Mesos master. While it seems like we > could > > > > > duplicate the logic in master/slave side to duplicate an instance > of > > > > > `MasterDetector`, it seems more natural if we could figure out a > > clean > > > > way > > > > > to share the existing detector instance in slave to the module. > > > > > > > > > > Any comment on this idea? > > > > > > > > > > Thanks. > > > > > > > > > > -- > > > > > Cheers, > > > > > > > > > > Zhitao Li > > > > > > > > > > > > > > > > > > > > > -- > > > > Avinash Sridharan, Mesosphere > > > > +1 (323) 702 5245 > > > > > > > > > > > > > > > > -- > > > Cheers, > > > > > > Zhitao Li > > > > > > > > > > > -- > > Avinash Sridharan, Mesosphere > > +1 (323) 702 5245 > > > -- Ilya Pronin
