On Wed, Apr 18, 2012 at 3:29 PM, Jesse Yates <[email protected]> wrote: > Its makes dealing with a lot of the zk operations easier to reason about. > Consider the 'create with parents' or the 'delete recursively' use cases - > non-transactional guarantees means we have a lot of extra logic for > checking under parents on creation, synchronous operations requiring extra > checking/notification etc; in short, its a lot of logic for things that > aren't, on a high-level, all that complicated (and are thought of > transactionally).
But how much do we really do these things? Isn't it only when we bootstrap a new cluster? > > These are the corner cases that are harder to find. For instance, what if > the parent is there, but the child has been put there - I need to leave a > watch and then do an update when children changed, and then read in all the > children, which could be actually more than I'm looking for, which leads to > a bunch of extra IO. If you can be sure that this goes away, we can get > much simpler code - notify for the node and then read the children. If the > children aren't there, they won't show up. this is just one possible > example, but others can also be constructed. You mean in the middle of a recursive op? > > It also can enable some really nice, new use cases. For instance, I've > discussed doing a Paxos-like mechanism that is far more easily enabled by > having transactional read-modify-write semantics for things like dynamic > table configuration (not proposing this as a jira (yet), and don't want to > get into if its a good idea, but rather just as an example for possible new > uses). ZK already has version-checking setData() calls which is enough for optimistic concurrency... I don't get the "paxos-like mechanism" here when ZK is already giving us consensus and atomic CAS. -Todd > > On Wed, Apr 18, 2012 at 2:49 PM, Todd Lipcon <[email protected]> wrote: > >> ZK 3.4.x still isn't marked "stable", so I'm a little hesitant to move >> to requiring it just yet. >> >> What tangible benefit do we gain from the "transactional" methods? >> >> -Todd >> >> On Wed, Apr 18, 2012 at 1:48 PM, lars hofhansl <[email protected]> >> wrote: >> > +1 >> > >> > >> > >> > ----- Original Message ----- >> > From: Jesse Yates <[email protected]> >> > To: [email protected] >> > Cc: >> > Sent: Wednesday, April 18, 2012 1:33 PM >> > Subject: requiring zookeeper 3.4.x in 0.96 >> > >> > Hi all, >> > >> > I was working on some patches to make ZK a bit more sane using the >> > transaction operations recently added (in jira: >> > https://issues.apache.org/jira/browse/ZOOKEEPER-965 >> > < >> http://zookeeper.apache.org/doc/r3.4.3/api/org/apache/zookeeper/ZooKeeper.html#transaction%28%29 >> > >> > and in the api here: >> > >> http://zookeeper.apache.org/doc/r3.4.3/api/org/apache/zookeeper/ZooKeeper.html#transaction%28%29 >> ). >> > This takes care of a lot of race conditions, etc. that are inherently >> > present in zk. >> > >> > Currently we (afaik) are not requiring ZooKeeper v.3.4.x in production >> and >> > instead allow users to run down to at least 3.3.x with the hbase's 3.4.x >> > client. Adding in transaction support will break this cross-version >> > functionality without a way of falling back as ZK doesn't (afaik) support >> > client-side look up of the server's version. Further, the fallback >> > functionality would be very dangerous if we assume transactions but fall >> > back to the non-transactional method; if we don't assume transactions, >> then >> > the transactional method is just wasted effort. >> > >> > I'm proposing that in the singularity (0.96), when we move to requiring >> > Hadoop v1.0+ we also make it a requirement that people run ZooKeeper >> v3.4+ >> > >> > Thoughts? >> > >> > Thanks! >> > >> > ------------------- >> > Jesse Yates >> > 240-888-2200 >> > @jesse_yates >> > jyates.github.com >> > >> >> >> >> -- >> Todd Lipcon >> Software Engineer, Cloudera >> -- Todd Lipcon Software Engineer, Cloudera
