Hi guys, Recently I open a question[1] on StackOverflow about leader election with ZooKeeper high-availability backend. It puzzles me for some days and it would be really help if you can take a look or even give some thoughts.
Copy the content to mailing list: Spark uses Curator#LeaderLatch for leader election. And PersistenceEngine for persistent. I'd like to know what if an old leader lost its leadership but before got notified, a new leader started to work, in which case both of the two contenders regarded themselves as the leader and wrote on zookeeper. Isn't it a race condition if the new leader just create a znode on zookeeper but the old one, which was no longer the leader, delete it. Best, tison. [1] https://stackoverflow.com/questions/55380498/how-spark-coordinates-multi-contender-race-on-writing-zookeeper