Re: Lock example
Thanks Mahadev, It's good for this confirmation as this is what I ended up doing. On Mon, Sep 13, 2010 at 11:33 PM, Mahadev Konar wrote: > Hi Tim, > The lock recipe you mention is supposed to avoid her affect and prevent > starvation (though it has bugs :)). > Are you looking for something like that or just a simple lock and unlock > that doesn't have to worry abt the above issues. > If that's the case then just doing an ephemeral create and delete should give > you your lock and unlock recipes. > > > Thanks > mahadev > > > On 9/8/10 9:58 PM, "Tim Robertson" wrote: > > Hi all, > > I am new to ZK and using the queue and lock examples that come with > zookeeper but have run into ZOOKEEPER-645 with the lock. > I have several JVMs each keeping a long running ZK client and the > first JVM (and hence client) does not respect the locks obtained by > subsequent clients - e.g. the first client always manages to get the > lock even if another client holds it. > > Before I start digging, I thought I'd ask if anyone has a simple lock > implemented they might share? My needs are simply to lock a URL to > indicate that it is being worked on, so that I don't hammer my > endpoints with multiple clients. > > Thanks for any advice, > Tim > >
Re: Understanding ZooKeeper data file management and LogFormatter
Hi Vishal, Usually the default retention policy is safe enough for operations. http://hadoop.apache.org/zookeeper/docs/r3.1.1/zookeeperAdmin.html Gives you an overview of how to use the purging library in zookeeper. Thanks mahadev On 9/8/10 12:01 PM, "Vishal K" wrote: > Hi All, > > Can you please share your experience regarding ZK snapshot retention and > recovery policies? > > We have an application where we never need to rollback (i.e., revert back to > a previous state by using old snapshots). Given this, I am trying to > understand under what circumstances would we ever need to use old ZK > snapshots. I understand a lot of these decisions depend on the application > and amount of redundancy used at every level (e.g,. RAID level where the > snapshots are stored etc) in the product. To simplify the discussion, I > would like to rule out any application characteristics and focus mainly on > data consistency. > > - Assuming that we have a 3 node cluster I am trying to figure out when > would I really need to use old snapshot files. With 3 nodes we already have > at least 2 servers with consistent database. If I loose files on one of the > servers, I can use files from the other. In fact, ZK server join will take > care of this. I can remove files from a faulty node and reboot that node. > The faulty node will sync with the leader. > > - The old files will be useful if the current snapshot and/or log files are > lost or corrupted on all 3 servers. If the loss is due to a disaster (case > where we loose all 3 servers), one would have to keep the snapshots on some > external storage to recover. However, if the current snapshot file is > corrupted on all 3 servers, then the most likely cause would be a bug in ZK. > In which case, how can I trust the consistency of the old snapshots? > > - Given a set of snapshots and log files, how can I verify the correctness > of these files? Example, if one of the intermediate snapshot file is > corrupt. > > - The Admin's guide says "Using older log and snapshot files, you can look > at the previous state of ZooKeeper servers and even restore that state. The > LogFormatter class allows an administrator to look at the transactions in a > log." * *Is there a tool that does this for the admin? The LogFormatter > only displays the transactions in the log file. > > - Has anyone ever had to play with the snapshot files in production? > > Thanks in advance. > > Regards, > -Vishal >
Re: Lock example
Hi Tim, The lock recipe you mention is supposed to avoid her affect and prevent starvation (though it has bugs :)). Are you looking for something like that or just a simple lock and unlock that doesn't have to worry abt the above issues. If that's the case then just doing an ephemeral create and delete should give you your lock and unlock recipes. Thanks mahadev On 9/8/10 9:58 PM, "Tim Robertson" wrote: Hi all, I am new to ZK and using the queue and lock examples that come with zookeeper but have run into ZOOKEEPER-645 with the lock. I have several JVMs each keeping a long running ZK client and the first JVM (and hence client) does not respect the locks obtained by subsequent clients - e.g. the first client always manages to get the lock even if another client holds it. Before I start digging, I thought I'd ask if anyone has a simple lock implemented they might share? My needs are simply to lock a URL to indicate that it is being worked on, so that I don't hammer my endpoints with multiple clients. Thanks for any advice, Tim
Re: Receiving create events for self with synchronous create
Hi Todd, Sorry for my late response. I had marked this email to respond but couldn't find the time :). Did you figure this out? It mostly looks like that as soon as you set a watch on /follower, some other node instantly creates another child of /follower? Could that be the case? Thanks mahadev On 8/26/10 8:09 PM, "Todd Nine" wrote: Sure thing. The FollowerWatcher class is instantiated by the IClusterManager implementation.It then performs the following FollowerWatcher.init() which is intended to do the following. 1. Create our follower node so that other nodes know we exist at path "/com/spidertracks/aviator/cluster/follower/10.0.1.1" where the last node is an ephemeral node with the internal IP address of the node. These are lines 67 through 72. 2. Signal to the clusterManager that the cluster has changed (line 79). Ultimately the clusterManager will perform a barrier for partitioning data ( a separate watcher) 3. Register a watcher to receive all future events on the follower path "/com/spidertracks/aviator/cluster/follower/" line 81. Then we have the following characteristics in the watcher 1. If a node has been added or deleted from the children of "/com/spidertracks/aviator/cluster/follower" then continue. Otherwise, ignore the event. Lines 33 through 44 2. If this was an event we should process our cluster has changed, signal to the CusterManager that a node has either been added or removed. line 51. I'm trying to encapsulate the detection of additions and deletions of child nodes within this Watcher. All other events that occur due to a node being added or deleted should be handled externally by the clustermanager. Thanks, Todd On Thu, 2010-08-26 at 19:26 -0700, Mahadev Konar wrote: Hi Todd, The code that you point to, I am not able to make out the sequence of steps. Can you be more clear on what you are trying to do in terms of zookeeper api? Thanks mahadev On 8/26/10 5:58 PM, "Todd Nine" wrote: Hi all, I'm running into a strange issue I could use a hand with. I've implemented leader election, and this is working well. I'm now implementing a follower queue with ephemeral nodes. I have an interface IClusterManager which simply has the api "clusterChanged". I don't care if nodes are added or deleted, I always want to fire this event. I have the following basic algorithm. init Create a path with "/follower/"+mynode name fire the clusterChangedEvent Watch set the event watcher on the path "/follower". watch: reset the watch on "/follower" if event is not a NodeDeleted or NodeCreated, ignore fire the clustermanager event this seems pretty straightforward. Here is what I'm expecting 1. Create my node path 2. fire the clusterChanged event 3. Set watch on "/follower" 4. Receive watch events for changes from any other nodes. What's actually happening 1. Create my node path 2. fire the clusterChanged event 3. Set Watch on "/follower" 4. Receive watch event for node created in step 1 5. Receive future watch events for changes from any other nodes. Here is my code. Since I set the watch after I create the node, I'm not expecting to receive the event for it. Am I doing something incorrectly in creating my watch? Here is my code. http://pastebin.com/zDXgLagd Thanks, Todd
