Re: Leader election and leader operation based on zookeeper

2019-09-20 Thread Zili Chen
Hi Jordan, Thanks for your pointing out. However, I'm not clear about lock strategy of Curator. Is it possible that getZookeeperClient().getZooKeeper() concurrent with a session expire and re-instance ZK client(thus I get the wrong session id)? Furthermore, even if I get the session id, check it

Re: Leader election and leader operation based on zookeeper

2019-09-20 Thread Jordan Zimmerman
> It seems Curator does not expose session id you can always access the ZooKeeper handle directly to get the session ID: CuratorFramework curator = ... curator.getZookeeperClient().getZooKeeper() -JZ > On Sep 20, 2019, at 10:21 PM, Zili Chen wrote: > > >>I am assuming the "write operation" he

Re: Leader election and leader operation based on zookeeper

2019-09-20 Thread Zili Chen
>>I am assuming the "write operation" here is write to ZooKeeper Yes. >>Looks like contender-1 was not reusing same ZooKeeper client object, so this explains how the previous supposed to be fail operation succeeds? Yes. Our communication to ZK is based on Curator, which will re-instance a client

Re: Ephemeral znode deleted infers session expired?

2019-09-20 Thread Michael Han
>> I'd like to know whether an ephemeral znode deleted infers its corresponding session expired. Yes as far as I know - assuming no one else was messing up with the same ephemeral node. On Thu, Sep 19, 2019 at 7:39 AM Zili Chen wrote: > Of course it is ensured that no other operations delete th

Re: Leader election and leader operation based on zookeeper

2019-09-20 Thread Michael Han
>> thus contender-1 commit a write operation even if it is no longer the leader I am assuming the "write operation" here is write to ZooKeeper (as opposed to write to an external storage system)? If so: >> contender-1 recovers from full gc, before it reacts to revoke leadership event, txn-1 retri

Leader election and leader operation based on zookeeper

2019-09-20 Thread Zili Chen
Hi ZooKeepers, Recently there is an ongoing refactor[1] in Flink community aimed at overcoming several inconsistent state issues on ZK we have met. I come here to share our design of leader election and leader operation. For leader operation, it is operation that should be committed only if the co