Hi Raghvendra and Spark users, I also have trouble activating my stand by master when my first master is shutdown (via a ./sbin/stop-master.sh or via a instance shut down) and just want to share with you my thoughts.
To answer your question Raghvendra, in *spark-env.sh*, if 2 IPs are set for SPARK_MASTER_IP(SPARK_MASTER_IP='W.X.Y.Z,A.B.C.D'), the standalone cluster cannot be launched. So I only use only one IP there, as the Spark context can know other masters with a other way, as written in the Standalone Zookeeper HA <http://spark.apache.org/docs/latest/spark-standalone.html#standby-masters-with-zookeeper> doc, "you might start your SparkContext pointing to spark://host1:port1,host2:port2" In my opinion, we should not have to set a SPARK_MASTER_IP as this is stored in ZooKeeper : you can launch multiple Masters in your cluster connected to the same > ZooKeeper instance. One will be elected “leader” and the others will remain > in standby mode. When starting up, an application or Worker needs to be able to find and > register with the current lead Master. Once it successfully registers, > though, it is “in the system” (i.e., stored in ZooKeeper). - http://spark.apache.org/docs/latest/spark-standalone.html#standby-masters-with-zookeeper As I understand it, after a ./sbin/stop-master.sh on both master, a master will be elected, and the other will be stand by. To launch the workers, we can use ./sbin/start-slave.sh spark://MASTER_ELECTED_IP:7077 I don't think if we can use the ./sbin/start-all.sh that use the salve file to launch workers and masters as we cannot set 2 master IPs inside spark-env.sh My SPARK_DAEMON_JAVA_OPTS content : SPARK_DAEMON_JAVA_OPTS='-Dspark.deploy.recoveryMode="ZOOKEEPER" > -Dspark.deploy.zookeeper.url="ZOOKEEPER_IP:2181" > -Dspark.deploy.zookeeper.dir="/spark"' A good thing to check if everything went OK is the folder /spark on the ZooKeeper server. I could not find it on my server. Thanks for reading, Paul 2016-01-19 22:12 GMT+01:00 Raghvendra Singh <raghvendra.ii...@gmail.com>: > Hi, there is one question. In spark-env.sh should i specify all masters > for parameter SPARK_MASTER_IP. I've set SPARK_DAEMON_JAVA_OPTS already > with zookeeper configuration as specified in spark documentation. > > Thanks & Regards > Raghvendra > > On Wed, Jan 20, 2016 at 1:46 AM, Raghvendra Singh < > raghvendra.ii...@gmail.com> wrote: > >> Here's the complete master log on reproducing the error >> http://pastebin.com/2YJpyBiF >> >> Regards >> Raghvendra >> >> On Wed, Jan 20, 2016 at 12:38 AM, Raghvendra Singh < >> raghvendra.ii...@gmail.com> wrote: >> >>> Ok I Will try to reproduce the problem. Also I don't think this is an >>> uncommon problem I am searching for this problem on Google for many days >>> and found lots of questions but no answers. >>> >>> Do you know what kinds of settings spark and zookeeper allow for >>> handling time outs during leader election etc. When one is down. >>> >>> Regards >>> Raghvendra >>> On 20-Jan-2016 12:28 am, "Ted Yu" <yuzhih...@gmail.com> wrote: >>> >>>> Perhaps I don't have enough information to make further progress. >>>> >>>> On Tue, Jan 19, 2016 at 10:55 AM, Raghvendra Singh < >>>> raghvendra.ii...@gmail.com> wrote: >>>> >>>>> I currently do not have access to those logs but there were only about >>>>> five lines before this error. They were the same which are present usually >>>>> when everything works fine. >>>>> >>>>> Can you still help? >>>>> >>>>> Regards >>>>> Raghvendra >>>>> On 18-Jan-2016 8:50 pm, "Ted Yu" <yuzhih...@gmail.com> wrote: >>>>> >>>>>> Can you pastebin master log before the error showed up ? >>>>>> >>>>>> The initial message was posted for Spark 1.2.0 >>>>>> Which release of Spark / zookeeper do you use ? >>>>>> >>>>>> Thanks >>>>>> >>>>>> On Mon, Jan 18, 2016 at 6:47 AM, doctorx <raghvendra.ii...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> I am facing the same issue, with the given error >>>>>>> >>>>>>> ERROR Master:75 - Leadership has been revoked -- master shutting >>>>>>> down. >>>>>>> >>>>>>> Can anybody help. Any clue will be useful. Should i change something >>>>>>> in >>>>>>> spark cluster or zookeeper. Is there any setting in spark which can >>>>>>> help me? >>>>>>> >>>>>>> Thanks & Regards >>>>>>> Raghvendra >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> View this message in context: >>>>>>> http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-0-standalone-ha-zookeeper-tp21308p25994.html >>>>>>> Sent from the Apache Spark User List mailing list archive at >>>>>>> Nabble.com. >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>>>>> For additional commands, e-mail: user-h...@spark.apache.org >>>>>>> >>>>>>> >>>>>> >>>> >> >
