Thanks for the response. But no this does not answer the question.
The question was: Is there a way (via some API call) to query the number and type of daemons currently running in the Spark cluster. Regards On Sun, Apr 26, 2015 at 10:12 AM, ayan guha <guha.a...@gmail.com> wrote: > In my limited understanding, there must be single "leader" master in > the cluster. If there are multiple leaders, it will lead to unstable > cluster as each masters will keep scheduling independently. You should use > zookeeper for HA, so that standby masters can vote to find new leader if > the primary goes down. > > Now, you can still have multiple masters running as leaders but > conceptually they should be thought as different clusters. > > Regarding workers, they should follow their master. > > Not sure if this answers your question, as I am sure you have read the > documentation thoroughly. > > Best > Ayan > > On Sun, Apr 26, 2015 at 6:31 PM, James King <jakwebin...@gmail.com> wrote: > >> If I have 5 nodes and I wish to maintain 1 Master and 2 Workers on each >> node, so in total I will have 5 master and 10 Workers. >> >> Now to maintain that setup I would like to query spark regarding the >> number Masters and Workers that are currently available using API calls and >> then take some appropriate action based on the information I get back, like >> restart a dead Master or Worker. >> >> Is this possible? does Spark provide such API? >> > > > > -- > Best Regards, > Ayan Guha >
