What is going on with this thread over on dev@zookeeper? Bringing it to the attention of people over here.
---------- Forwarded message ---------- From: Ted Dunning <ted.dunn...@gmail.com> Date: Fri, Jan 17, 2014 at 2:41 PM Subject: Re: Where are we in ZOOKEEPER-1416 To: "d...@zookeeper.apache.org" <d...@zookeeper.apache.org> My reference here is to the comments a ways up thread. Kishore and I clearly agree completely that idempotency and dealing with the state as it is right now are the keys to correct design. On Fri, Jan 17, 2014 at 2:14 PM, Ted Dunning <ted.dunn...@gmail.com> wrote: > > That comment indicates a lack of understanding of ZK, not a bug in ZK. > > You don't lose state transitions if you read new state at the same time > you set the new watch. > > Likewise, it is simply a product of bad design to have a problem with > asynchronous notification. Changes on other machines *are* asynchronous so > anybody who can't handle that is inherently denying reality. If you want > to inject the notifications into a sequential view of an event stream, that > is trivial to do. > > Systems that depend on transition notification are generally not as robust > as systems that depend on current state. Building a cluster manager works > better if the master is notified that a change has happened, but then > simply deals with the situation as it stands. > > As an analog, imagine that you have a system that shows a number x and a > second system that is supposed to show an echo of that number. > > Design A is notified of changes to x in the form of deltas. If there is > ever an error in handling events, the echo will be off forever. The error > that causes the delta to be dropped could be notification or a coding error > or a misunderstanding of how parallel systems work. For instance, the > InterruptedException might not be handled right. > > Design B is notified of changes to x and whenever a change happens, the > second system simply goes and reads the new state. Errors will be quickly > corrected. > > It sounds like the original poster is trying to build something like > Design A when they should be building Design B. > > > > > > On Fri, Jan 17, 2014 at 12:34 PM, Ted Yu <yuzhih...@gmail.com> wrote: > >> HBASE-5487 is also related. >> >> The discussion there is very long. Below is an excerpt from Honghua: >> >> too many tricky scenarios/bugs due to ZK watch is one-time(which can >> result >> in missed state transition) and the notification/process is >> asyncronous(which can lead to delayed/non-update-to-date state in master >> memory). >> >> Cheers >> >> >> On Fri, Jan 17, 2014 at 11:25 AM, Ted Yu <yuzhih...@gmail.com> wrote: >> >> > Hi, Flavio: >> > HBASE-8365 is one such case. >> > >> > Let me search around for other related discussion. >> > >> > >> > On Fri, Jan 17, 2014 at 11:17 AM, Flavio Junqueira < >> fpjunque...@yahoo.com>wrote: >> > >> >> Hi Ted, >> >> >> >> Can you provide more detail on how the precise deltas could make it >> more >> >> robust? >> >> >> >> -Flavio >> >> >> >> -----Original Message----- >> >> From: "Ted Yu" <yuzhih...@gmail.com> >> >> Sent: 17/01/2014 17:25 >> >> To: "d...@zookeeper.apache.org" <d...@zookeeper.apache.org> >> >> Subject: Re: Where are we in ZOOKEEPER-1416 >> >> >> >> Having the ability to know exact deltas would help make HBase region >> >> assignment more robust. >> >> >> >> Cheers >> >> >> >> >> >> >> >> On Fri, Jan 17, 2014 at 9:13 AM, kishore g <g.kish...@gmail.com> >> wrote: >> >> >> >> > I agree with you, I like the side effect and in fact I would prefer >> to >> >> have >> >> > one notification for all changes under a parent node. >> >> > >> >> > However, Hao is probably asking for ability to know exact deltas. >> >> > >> >> > >> >> > On Fri, Jan 17, 2014 at 8:15 AM, FPJ <fpjunque...@yahoo.com> wrote: >> >> > >> >> > > We don't need to have a mapping between every change and a >> >> notification. >> >> > If >> >> > > there are 2+ changes between notifications, you'll be able to >> observe >> >> it >> >> > by >> >> > > reading the ZK state. In fact, one nice side-effect is that we >> reduce >> >> the >> >> > > number of notifications when there are many concurrent changes. >> >> > > >> >> > > The only situation I can see it being necessary is the one in >> which we >> >> > need >> >> > > to know precisely the changes and we haven't cached a previous >> >> version of >> >> > > the state. >> >> > > >> >> > > -Flavio >> >> > > >> >> > > > -----Original Message----- >> >> > > > From: kishore g [mailto:g.kish...@gmail.com] >> >> > > > Sent: 17 January 2014 16:06 >> >> > > > To: d...@zookeeper.apache.org >> >> > > > Subject: Re: Where are we in ZOOKEEPER-1416 >> >> > > > >> >> > > > I think Hao is pointing out that there is no way to see every >> change >> >> > > > (delta) that happened to a znode. Consider 2 changes A,B in quick >> >> > > > succession. When client gets notified of A and before setting the >> >> watch >> >> > > the >> >> > > > change B has occurred on the server side. This means the client >> >> cannot >> >> > > know >> >> > > > the delta A. Client can only read the state after change B is >> >> applied. >> >> > > > >> >> > > > Implementing the concept of Persistent watcher guarantees that >> >> client >> >> > if >> >> > > > notified after every change. >> >> > > > >> >> > > > This is a nice to have feature but I dont understand the >> >> requirement in >> >> > > Hbase >> >> > > > where this is needed. Hao, can you shed more light on how this >> >> would be >> >> > > > useful for HBase (to act like state machine) >> >> > > > >> >> > > > thanks, >> >> > > > Kishore G >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > On Fri, Jan 17, 2014 at 5:18 AM, FPJ <fpjunque...@yahoo.com> >> wrote: >> >> > > > >> >> > > > > But you don't really miss events, you'll see them when you read >> >> the >> >> > ZK >> >> > > > > state. If you follow the pattern I described, you're supposed >> to >> >> > > > > observe all changes. Perhaps I'm missing some concrete use case >> >> you >> >> > > > > have mind. >> >> > > > > >> >> > > > > -Flavio >> >> > > > > >> >> > > > > > -----Original Message----- >> >> > > > > > From: 陈迪豪 [mailto:chendi...@xiaomi.com] >> >> > > > > > Sent: 17 January 2014 13:03 >> >> > > > > > To: d...@zookeeper.apache.org >> >> > > > > > Subject: RE: Where are we in ZOOKEEPER-1416 >> >> > > > > > >> >> > > > > > No, it's not complicated. But for the people who don't >> >> understand >> >> > zk >> >> > > > > deeply, >> >> > > > > > they would easily ignore the fact that they would miss >> events in >> >> > > > > > some >> >> > > > > way. >> >> > > > > > Moreover, I think providing persistent watch is good for >> >> developers >> >> > > > > > to >> >> > > > > build >> >> > > > > > the "state-machine" application. Actually, HBase suffer from >> >> > missing >> >> > > > > > the intermediate state when use zk to store the data. >> >> > > > > > >> >> > > > > > If the feature is implemented, I would like to see the patch >> and >> >> > > > > > consider >> >> > > > > if it >> >> > > > > > can be used for us. >> >> > > > > > >> >> > > > > > ________________________________________ >> >> > > > > > From: Flavio Junqueira [fpjunque...@yahoo.com] >> >> > > > > > Sent: Friday, January 17, 2014 8:12 PM >> >> > > > > > To: d...@zookeeper.apache.org >> >> > > > > > Subject: RE: Where are we in ZOOKEEPER-1416 >> >> > > > > > >> >> > > > > > My take is that persistent subscriptions add complexity and >> are >> >> not >> >> > > > > strictly >> >> > > > > > necessary. You can follow this pattern of setting a watch, >> >> reading >> >> > > > > > the >> >> > > > > state >> >> > > > > > upon a notification and setting a new watch. Why do you feel >> >> that's >> >> > > > > > complicated? >> >> > > > > > >> >> > > > > > -Flavio >> >> > > > > > >> >> > > > > > -----Original Message----- >> >> > > > > > From: 陈迪豪 [mailto:chendi...@xiaomi.com] >> >> > > > > > Sent: Friday, January 17, 2014 3:13 AM >> >> > > > > > To: d...@zookeeper.apache.org >> >> > > > > > Subject: Where are we in ZOOKEEPER-1416 >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > Persistent watch and implementing the feature to act like >> "state >> >> > > > machine" >> >> > > > > > which is mentioned in >> >> > > > > > ZOOKEEPER-153< >> https://issues.apache.org/jira/browse/ZOOKEEPER- >> >> > > > 153>, >> >> > > > > > would be great for ZooKeeper user. I think HBase would like >> to >> >> know >> >> > > > > > all >> >> > > > > the >> >> > > > > > change in zk rather than missing kind of events. >> >> > > > > > >> >> > > > > > So, would we continue developing these features? It's also a >> >> little >> >> > > > > > complicated to develop with zk and I think there're lots of >> >> things >> >> > > > > > to >> >> > > > > improve. >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > >> >> > > >> >> > >> >> >> > >> > >> > > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
