Re: Dealing with session expired

2009-02-12 Thread Patrick Hunt
Tom, you might try changing the log4j default log level to DEBUG for the rootlogger and appender if you have not already done so (servers and clients both). You'll get more information to aid debugging if it does occur again. http://hadoop.apache.org/zookeeper/docs/r3.0.1/zookeeperAdmin.html#sc

Hadoop User Group Meeting (Bay Area) 2/18

2009-02-12 Thread Ajay Anand
The next Bay Area Hadoop User Group meeting is scheduled for Wednesday, February 18th at Yahoo! 2811 Mission College Blvd, Santa Clara, Building 2, Training Rooms 5 & 6 from 6:00-7:30 pm. Agenda: Fair Scheduler for Hadoop - Matei Zaharia Interfacing with MySQL - Aaron Kimball Registration

Re: Dealing with session expired

2009-02-12 Thread Tom Nichols
On Thu, Feb 12, 2009 at 4:11 PM, Benjamin Reed wrote: > idleness is not a problem. the client library sends heartbeats to keep the > session alive. the client library will also handle reconnects automatically > if a server dies. That's odd then that I'm seeing this problem. I have a local, 3-n

RE: Dealing with session expired

2009-02-12 Thread Benjamin Reed
idleness is not a problem. the client library sends heartbeats to keep the session alive. the client library will also handle reconnects automatically if a server dies. since session expiration really is a rare catastrophic event. (or at least it should be.) it is probably easiest to deal with

Re: Dealing with session expired

2009-02-12 Thread Tom Nichols
I'm using a timeout of 5000ms. Now let me ask this: Suppose all of my clients are waiting on some external event -- not ZooKeeper -- so they are all idle and are not touching ZK nodes, nor are they calling exists, getChildren, etc etc. Can that idleness cause session expiry? I'm running a local

Re: Dealing with session expired

2009-02-12 Thread Patrick Hunt
Regardless of frequency Tom's code still has to handle this situation. I would suggest that the "two classes" Tom is referring to in his mail, the ones that use ZK client object, should either be able to "reinitialize" with a new zk session, or they themselves should be discarded and new insta

Re: Dealing with session expired

2009-02-12 Thread Patrick Hunt
Ephemerals and watches are maintained across disconnect/reconnect btw the client and server however session expiration (or closing the session explicitly) will trigger deletion of ephemeral nodes associated with the session. Right - once the session is expired the id is invalid. You need to c

Re: Dealing with session expired

2009-02-12 Thread Mahadev Konar
Hi Tom, The session expired event means that the the server expired the client and that means the watches and ephemrals will go away for that node. How are you running your zookeeper quorum? Session expiry event should be really rare event . If you have a quorum of servers it should rarely happen

Re: Dealing with session expired

2009-02-12 Thread Tom Nichols
So if a session expires, my ephemeral nodes and watches have already disappeared? I suppose creating a new ZK instance with the old session ID would not do me any good in that case. Correct? Thanks. -Tom On Thu, Feb 12, 2009 at 2:12 PM, Mahadev Konar wrote: > Hi Tom, > We prefer to discard

Re: Dealing with session expired

2009-02-12 Thread Mahadev Konar
Hi Tom, We prefer to discard the zookeeper instance if a session expires. Maintaining a one to one relationship between a client handle and a session makes it much simpler for users to understand the existence and disappearance of ephemeral nodes and watches created by a zookeeper client. thanks

Dealing with session expired

2009-02-12 Thread Tom Nichols
I've come across the situation where a ZK instance will have an expired connection and therefore all operations fail. Now AFAIK the only way to recover is to create a new ZK instance with the old session ID, correct? Now, my problem is, the ZK instance may be shared -- not between threads -- but