Re: Simpler ZooKeeper event interface....

2009-01-07 Thread Vinod Johnson
Hiram Chirino wrote: Knowing about a disconnection may be important to some apps. For example if an app uses ZK for leader election, and the leader gets disconnected from ZK, he should give up being the leader, since a different leader may get elected while he is disconnected from ZK. Is

Re: Reconnecting to another host on failure but before session expires...

2009-01-07 Thread Vinod Johnson
Kevin Burton wrote: Crazy, I don't know how I missed that... Wouldn't it be cleaner to specify this as a ListString of host:port names? If API cleanup was being considered, my inclination would have been ListInetSocketAddress.

Re: Reconnecting to another host on failure but before session expires...

2009-01-07 Thread Patrick Hunt
There's also been interest in having a chroot type capability as part of the connect string: host:port/app/abc,... where the client's session would be rooted at /app/abc rather than / This is very useful in multi-tenant situations (more than 1 app sharing a zk cluster). Patrick Benjamin

Re: Simpler ZooKeeper event interface....

2009-01-07 Thread Patrick Hunt
Kevin Burton wrote: 3) it's possible for your code to get notified of a change, but never process the change. This might happen if: a) a node changed watch fires b) your client code runs an async getData c) you are disconnected from the server Also, this seems very confusing... If I run

Re: Simpler ZooKeeper event interface....

2009-01-07 Thread Mahadev Konar
Hi Vinod, I think what Ben meant was this-- The client will never know of a session expiration until and unless its connected to one of the servers. So the leader cannot demote itself since its connected to one of the servers. It might have lost its session (which all the others except itself

Re: ouch, zookeeper infinite loop

2009-01-07 Thread Mahadev Konar
The version of Jute we use is really an ancient version of recordio ser/deser library in hadoop. We do want to move to some better(versioned/fast/well accepted) ser/deser library. mahadev On 1/7/09 12:08 PM, Kevin Burton bur...@spinn3r.com wrote: Ah... you think it was because it was empty?

Re: Sending data during NodeDataChanged or NodeCreated

2009-01-07 Thread Kevin Burton
On Wed, Jan 7, 2009 at 9:25 AM, Benjamin Reed br...@yahoo-inc.com wrote: This is the behavior we had when we first implemented the API, and in every case where people used the information there was a bug. it is virtually impossible to use correctly. In general I'm all for giving people rope,

Re: Does session expiration only happen during total ensemble failure or network split?

2009-01-07 Thread Mahadev Konar
Kevin, the case you mention, the session is not really expired unless the quorum decides to expire it. So the client assuming that the session expired would be wrong to say. It is possible that as soon as you bring up the servers, the client reconnects with the same session and the session is

Re: Sending data during NodeDataChanged or NodeCreated

2009-01-07 Thread Adam Rosien
On Wed, Jan 7, 2009 at 12:46 PM, Kevin Burton bur...@spinn3r.com wrote: On Wed, Jan 7, 2009 at 9:25 AM, Benjamin Reed br...@yahoo-inc.com wrote: This is the behavior we had when we first implemented the API, and in every case where people used the information there was a bug. it is virtually

Re: Simpler ZooKeeper event interface....

2009-01-07 Thread Vinod Johnson
Mahadev Konar wrote: Hi Vinod, I think what Ben meant was this-- The client will never know of a session expiration until and unless its connected to one of the servers. So the leader cannot demote itself since its connected to one of the servers. It might have lost its session (which all the

Re: Simpler ZooKeeper event interface....

2009-01-07 Thread Patrick Hunt
To say that it will never return is not correct. The client will be notified of connectionloss in the callback, however the client will not know if the operation was successful (from point of view of the server) or not. Patrick Kevin Burton wrote: On Wed, Jan 7, 2009 at 11:12 AM, Mahadev

Re: Does session expiration only happen during total ensemble failure or network split?

2009-01-07 Thread Patrick Hunt
Mahadev Konar wrote: Why would you want the session to expire if all the servers are down (which should not happen unless you kill all the nodes or the datacenter is down) ? A more likely case is that the client port on the switch dies and the client is partitioned from the servers...

Re: ouch, zookeeper infinite loop

2009-01-07 Thread Patrick Hunt
Whatever we do the changes should support having more than one marshaling format/version co-exist. Both for b/w compatibility as well as enabling different serialization mechanisms (jute or pbuffer or thrift or etch, etc...) Patrick Mahadev Konar wrote: The version of Jute we use is really

Re: ouch, zookeeper infinite loop

2009-01-07 Thread Kevin Burton
protocol buffers are nice... the variable int stuff is cool. Though is the protocol very complicated? Why not write a custom hand written Externalizable format? These frameworks are often more trouble than they're worth for small protocols. Kevin On Wed, Jan 7, 2009 at 2:46 PM, Patrick

Re: Simpler ZooKeeper event interface....

2009-01-07 Thread Vinod Johnson
I guess then I don't follow the leader election recipe. Is the following scenario possible in the leader election recipe: 1) Leader L is partitioned from the ensemble. 2) ZK servers expire its session. 3) Some other follower F now becomes a leader. 4) L and F form a split brain? I had

Re: Simpler ZooKeeper event interface....

2009-01-07 Thread Patrick Hunt
Kevin Burton wrote: Here's a good reason for each client to know it's session status (connected/disconnected/expired). Depending on the application, if L does not have a connected session to the ensemble it may need to be careful how it acts. connected/disconnected events are given out in

RE: Simpler ZooKeeper event interface....

2009-01-07 Thread Benjamin Reed
when you shutdown the full ensemble the session isn't expired. when things come back up your session will still be active. (it would be bad if the zk service could not survive the bounce of an ensembel.) you are way over thinking this and i fear you are not helping yourself with trying to