zookeeper is not a direct dependency of Spark.
Can you give a bit more detail on how the election / discovery of master
works ?
Cheers
On Thu, Jul 30, 2015 at 7:41 PM, Christophe Schmitz cofcof...@gmail.com
wrote:
Hi there,
I am trying to run a 3 node spark cluster where each nodes contains
Hi Ted,
Thanks for your reply. I think zookeeper is an optional dependency of
Spark. To enable it, I essentially use this flags on all my spark-env.sh:
SPARK_DAEMON_JAVA_OPTS=-Dspark.deploy.recoveryMode=ZOOKEEPER
-Dspark.deploy.zookeeper.url=my-zoo-ip:2181
and of course, I have my zookeeper
Hi there,
I am trying to run a 3 node spark cluster where each nodes contains a spark
worker and a spark maser. Election of the master happens via zookeeper.
The way I am configuring it is by (on each node) giving the IP:PORT of the
local master to the local worker, and I wish the worker could
