On Fri, Jan 17, 2014 at 2:53 PM, Andrew Purtell <apurt...@apache.org> wrote:
> What is going on with this thread over on dev@zookeeper? Bringing it to > the > attention of people over here. > > > Ted? St.Ack > ---------- Forwarded message ---------- > From: Ted Dunning <ted.dunn...@gmail.com> > Date: Fri, Jan 17, 2014 at 2:41 PM > Subject: Re: Where are we in ZOOKEEPER-1416 > To: "d...@zookeeper.apache.org" <d...@zookeeper.apache.org> > > > My reference here is to the comments a ways up thread. Kishore and I > clearly agree completely that idempotency and dealing with the state as it > is right now are the keys to correct design. > > > On Fri, Jan 17, 2014 at 2:14 PM, Ted Dunning <ted.dunn...@gmail.com> > wrote: > > > > > That comment indicates a lack of understanding of ZK, not a bug in ZK. > > > > You don't lose state transitions if you read new state at the same time > > you set the new watch. > > > > Likewise, it is simply a product of bad design to have a problem with > > asynchronous notification. Changes on other machines *are* asynchronous > so > > anybody who can't handle that is inherently denying reality. If you want > > to inject the notifications into a sequential view of an event stream, > that > > is trivial to do. > > > > Systems that depend on transition notification are generally not as > robust > > as systems that depend on current state. Building a cluster manager > works > > better if the master is notified that a change has happened, but then > > simply deals with the situation as it stands. > > > > As an analog, imagine that you have a system that shows a number x and a > > second system that is supposed to show an echo of that number. > > > > Design A is notified of changes to x in the form of deltas. If there is > > ever an error in handling events, the echo will be off forever. The > error > > that causes the delta to be dropped could be notification or a coding > error > > or a misunderstanding of how parallel systems work. For instance, the > > InterruptedException might not be handled right. > > > > Design B is notified of changes to x and whenever a change happens, the > > second system simply goes and reads the new state. Errors will be > quickly > > corrected. > > > > It sounds like the original poster is trying to build something like > > Design A when they should be building Design B. > > > > > > > > > > > > On Fri, Jan 17, 2014 at 12:34 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > >> HBASE-5487 is also related. > >> > >> The discussion there is very long. Below is an excerpt from Honghua: > >> > >> too many tricky scenarios/bugs due to ZK watch is one-time(which can > >> result > >> in missed state transition) and the notification/process is > >> asyncronous(which can lead to delayed/non-update-to-date state in master > >> memory). > >> > >> Cheers > >> > >> > >> On Fri, Jan 17, 2014 at 11:25 AM, Ted Yu <yuzhih...@gmail.com> wrote: > >> > >> > Hi, Flavio: > >> > HBASE-8365 is one such case. > >> > > >> > Let me search around for other related discussion. > >> > > >> > > >> > On Fri, Jan 17, 2014 at 11:17 AM, Flavio Junqueira < > >> fpjunque...@yahoo.com>wrote: > >> > > >> >> Hi Ted, > >> >> > >> >> Can you provide more detail on how the precise deltas could make it > >> more > >> >> robust? > >> >> > >> >> -Flavio > >> >> > >> >> -----Original Message----- > >> >> From: "Ted Yu" <yuzhih...@gmail.com> > >> >> Sent: 17/01/2014 17:25 > >> >> To: "d...@zookeeper.apache.org" <d...@zookeeper.apache.org> > >> >> Subject: Re: Where are we in ZOOKEEPER-1416 > >> >> > >> >> Having the ability to know exact deltas would help make HBase region > >> >> assignment more robust. > >> >> > >> >> Cheers > >> >> > >> >> > >> >> > >> >> On Fri, Jan 17, 2014 at 9:13 AM, kishore g <g.kish...@gmail.com> > >> wrote: > >> >> > >> >> > I agree with you, I like the side effect and in fact I would prefer > >> to > >> >> have > >> >> > one notification for all changes under a parent node. > >> >> > > >> >> > However, Hao is probably asking for ability to know exact deltas. > >> >> > > >> >> > > >> >> > On Fri, Jan 17, 2014 at 8:15 AM, FPJ <fpjunque...@yahoo.com> > wrote: > >> >> > > >> >> > > We don't need to have a mapping between every change and a > >> >> notification. > >> >> > If > >> >> > > there are 2+ changes between notifications, you'll be able to > >> observe > >> >> it > >> >> > by > >> >> > > reading the ZK state. In fact, one nice side-effect is that we > >> reduce > >> >> the > >> >> > > number of notifications when there are many concurrent changes. > >> >> > > > >> >> > > The only situation I can see it being necessary is the one in > >> which we > >> >> > need > >> >> > > to know precisely the changes and we haven't cached a previous > >> >> version of > >> >> > > the state. > >> >> > > > >> >> > > -Flavio > >> >> > > > >> >> > > > -----Original Message----- > >> >> > > > From: kishore g [mailto:g.kish...@gmail.com] > >> >> > > > Sent: 17 January 2014 16:06 > >> >> > > > To: d...@zookeeper.apache.org > >> >> > > > Subject: Re: Where are we in ZOOKEEPER-1416 > >> >> > > > > >> >> > > > I think Hao is pointing out that there is no way to see every > >> change > >> >> > > > (delta) that happened to a znode. Consider 2 changes A,B in > quick > >> >> > > > succession. When client gets notified of A and before setting > the > >> >> watch > >> >> > > the > >> >> > > > change B has occurred on the server side. This means the client > >> >> cannot > >> >> > > know > >> >> > > > the delta A. Client can only read the state after change B is > >> >> applied. > >> >> > > > > >> >> > > > Implementing the concept of Persistent watcher guarantees that > >> >> client > >> >> > if > >> >> > > > notified after every change. > >> >> > > > > >> >> > > > This is a nice to have feature but I dont understand the > >> >> requirement in > >> >> > > Hbase > >> >> > > > where this is needed. Hao, can you shed more light on how this > >> >> would be > >> >> > > > useful for HBase (to act like state machine) > >> >> > > > > >> >> > > > thanks, > >> >> > > > Kishore G > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > On Fri, Jan 17, 2014 at 5:18 AM, FPJ <fpjunque...@yahoo.com> > >> wrote: > >> >> > > > > >> >> > > > > But you don't really miss events, you'll see them when you > read > >> >> the > >> >> > ZK > >> >> > > > > state. If you follow the pattern I described, you're supposed > >> to > >> >> > > > > observe all changes. Perhaps I'm missing some concrete use > case > >> >> you > >> >> > > > > have mind. > >> >> > > > > > >> >> > > > > -Flavio > >> >> > > > > > >> >> > > > > > -----Original Message----- > >> >> > > > > > From: 陈迪豪 [mailto:chendi...@xiaomi.com] > >> >> > > > > > Sent: 17 January 2014 13:03 > >> >> > > > > > To: d...@zookeeper.apache.org > >> >> > > > > > Subject: RE: Where are we in ZOOKEEPER-1416 > >> >> > > > > > > >> >> > > > > > No, it's not complicated. But for the people who don't > >> >> understand > >> >> > zk > >> >> > > > > deeply, > >> >> > > > > > they would easily ignore the fact that they would miss > >> events in > >> >> > > > > > some > >> >> > > > > way. > >> >> > > > > > Moreover, I think providing persistent watch is good for > >> >> developers > >> >> > > > > > to > >> >> > > > > build > >> >> > > > > > the "state-machine" application. Actually, HBase suffer > from > >> >> > missing > >> >> > > > > > the intermediate state when use zk to store the data. > >> >> > > > > > > >> >> > > > > > If the feature is implemented, I would like to see the > patch > >> and > >> >> > > > > > consider > >> >> > > > > if it > >> >> > > > > > can be used for us. > >> >> > > > > > > >> >> > > > > > ________________________________________ > >> >> > > > > > From: Flavio Junqueira [fpjunque...@yahoo.com] > >> >> > > > > > Sent: Friday, January 17, 2014 8:12 PM > >> >> > > > > > To: d...@zookeeper.apache.org > >> >> > > > > > Subject: RE: Where are we in ZOOKEEPER-1416 > >> >> > > > > > > >> >> > > > > > My take is that persistent subscriptions add complexity and > >> are > >> >> not > >> >> > > > > strictly > >> >> > > > > > necessary. You can follow this pattern of setting a watch, > >> >> reading > >> >> > > > > > the > >> >> > > > > state > >> >> > > > > > upon a notification and setting a new watch. Why do you > feel > >> >> that's > >> >> > > > > > complicated? > >> >> > > > > > > >> >> > > > > > -Flavio > >> >> > > > > > > >> >> > > > > > -----Original Message----- > >> >> > > > > > From: 陈迪豪 [mailto:chendi...@xiaomi.com] > >> >> > > > > > Sent: Friday, January 17, 2014 3:13 AM > >> >> > > > > > To: d...@zookeeper.apache.org > >> >> > > > > > Subject: Where are we in ZOOKEEPER-1416 > >> >> > > > > > > >> >> > > > > > > >> >> > > > > > > >> >> > > > > > Persistent watch and implementing the feature to act like > >> "state > >> >> > > > machine" > >> >> > > > > > which is mentioned in > >> >> > > > > > ZOOKEEPER-153< > >> https://issues.apache.org/jira/browse/ZOOKEEPER- > >> >> > > > 153>, > >> >> > > > > > would be great for ZooKeeper user. I think HBase would like > >> to > >> >> know > >> >> > > > > > all > >> >> > > > > the > >> >> > > > > > change in zk rather than missing kind of events. > >> >> > > > > > > >> >> > > > > > So, would we continue developing these features? It's also > a > >> >> little > >> >> > > > > > complicated to develop with zk and I think there're lots of > >> >> things > >> >> > > > > > to > >> >> > > > > improve. > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > >> >> > > > >> >> > > >> >> > >> > > >> > > >> > > > > > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) >
