Hi, We are planning to install Azkaban in solo server mode on a 24 node cassandra cluster to be able to schedule spark jobs with intricate dependency chain. The problem, is since Cassandra has a no-SPOF architecture ie any node can become the master for the cluster, it creates the problem for Azkaban master since it's not a peer-peer architecture where any node can become the master. Only a single mode has to be master at any given time.
What are our options here? Are there any framworks or tools out there that would allow any application to run on a cluster of machines with high availablity? Should I be looking at something like zookeeper for this ? Or Mesos may be?
