zookeeper is not a direct dependency of Spark. Can you give a bit more detail on how the election / discovery of master works ?
Cheers On Thu, Jul 30, 2015 at 7:41 PM, Christophe Schmitz <cofcof...@gmail.com> wrote: > Hi there, > > I am trying to run a 3 node spark cluster where each nodes contains a > spark worker and a spark maser. Election of the master happens via > zookeeper. > > The way I am configuring it is by (on each node) giving the IP:PORT of the > local master to the local worker, and I wish the worker could autodiscover > the elected master automatically. > > But unfortunatly, only the local worker of the elected master registered > to the elected master. Why aren't the other worker getting to connect to > the elected master? > > The interessing thing is that if I kill the elected master and wait a bit, > then the new elected master sees all the workers! > > I am wondering if I am missing something to make this happens without > having to kill the elected master. > > Thanks! > > > PS: I am on spark 1.2.2 > >
