Re: [DISCUESS] PIP 57: Improve Broker's Zookeeper Session Timeout Handling

2020-02-24 Thread PengHui Li
Hi Joe, I will improve the PIP, and then send it back to the email thread soon. -Penghui 在 2020年2月25日 +0800 AM4:41,Sijie Guo ,写道: > On Mon, Feb 24, 2020 at 11:33 AM Joe F wrote: > > > Sjiie, Penghui, > > > > Thank you. Can we get the PIP to be a more detailed write-up ? I would > > like this PIP

Re: [DISCUESS] PIP 57: Improve Broker's Zookeeper Session Timeout Handling

2020-02-24 Thread Sijie Guo
On Mon, Feb 24, 2020 at 11:33 AM Joe F wrote: > Sjiie, Penghui, > > Thank you. Can we get the PIP to be a more detailed write-up ? I would > like this PIP to be more comprehensive > I think Penghui can work on improving this to make this PIP to be more comprehensive. It is good to start with

Re: [DISCUESS] PIP 57: Improve Broker's Zookeeper Session Timeout Handling

2020-02-24 Thread Joe F
Sjiie, Penghui, Thank you. Can we get the PIP to be a more detailed write-up ? I would like this PIP to be more comprehensive >>Hence we need to draw an agreement on understanding >>WHAT is actually guarantees the correctness in current Pulsar design. We >>then can move forward with a conclusio

Re: [DISCUESS] PIP 57: Improve Broker's Zookeeper Session Timeout Handling

2020-02-23 Thread Sijie Guo
Sorry for the late reply. Joe - There are two things I would like to clarify first. 1) I think you have a misunderstanding about the zookeeper lock "ephemeral znode" and bookkeeper/ML fencing. Let's step back to understand the current Pulsar's behavior first. - A zookeeper lock doesn't prevent a

Re: [DISCUESS] PIP 57: Improve Broker's Zookeeper Session Timeout Handling

2020-02-22 Thread Joe F
On Sat, Feb 22, 2020 at 6:28 PM PengHui Li wrote: > Hi, joe > > The fundamental correctness is guaranteed by the fencing mechanism > provided by Apache BookKeeper and the CAS operation provided by the > metadata storage. Both fencing and CAS operations will prevent two owners > updating data or m

Re: [DISCUESS] PIP 57: Improve Broker's Zookeeper Session Timeout Handling

2020-02-22 Thread PengHui Li
Hi, joe The fundamental correctness is guaranteed by the fencing mechanism provided by Apache BookKeeper and the CAS operation provided by the metadata storage. Both fencing and CAS operations will prevent two owners updating data or metadata at the same time. This default shutdown behavior isn

Re: [DISCUESS] PIP 57: Improve Broker's Zookeeper Session Timeout Handling

2020-02-22 Thread Joe F
My concerns are listed in the PR comments. A broker is allowed to operate on a (resource) bundle under a lock. When a broker loses its session, the lock ownership COULD be lost. The right thing at this point is to give up the resource and re-acquire it. ( In fact, shutdown is just a shortcut to do

[DISCUESS] PIP 57: Improve Broker's Zookeeper Session Timeout Handling

2020-02-21 Thread PengHui Li
Hi all, I have drafted a proposal for improving broker's Zookeeper session timeout handling. You can find at  https://github.com/apache/pulsar/wiki/PIP-57%3A-Improve-Broker%27s-Zookeeper-Session-Timeout-Handling Also I copy it to the email thread for easier to view. Any suggestions or ideas wel