Re: Exists Watch Triggered by Delete
No problem, we really appreciate your feedback -- keep it coming. Patrick Stu Hood wrote: Thanks a bunch Patrick! I really appreciate your work on ZooKeeper... you have made a tremendous impact on the project. I will try the ZOOKEEPER-221 patch. Thanks, Stu -Original Message- From: "Patrick Hunt" <[EMAIL PROTECTED]> Sent: Wednesday, November 12, 2008 6:37pm To: zookeeper-user@hadoop.apache.org Subject: Re: Exists Watch Triggered by Delete Stu Hood wrote: The comment you referenced in your original email is true - that code should never execute as the existsWatches list only contains watches for NONODE watch registrations (which obv couldn't be deleted since it doesn't exist). So I am experiencing a bug then? Wow, this is a dumb mistake. The log message you pointed to is being output _every time_ this code is called. There needs to be a conditional on the log message to check the result of the remove call. I have entered a JIRA and have already submitted a patch: https://issues.apache.org/jira/browse/ZOOKEEPER-221 So unfortunately we can't tell if this is the source of the issue you are seeing. Can you run with the patch attached to ZOOKEEPER-221? It may be better not to let the programmer know about those two lists, and to see if the abstraction can be improved instead. Sometimes I feel that I have to know too much about the internal working of ZooKeeper to use its API. That's a good point. We're working to improve the docs & code, I've made a note of it. Perhaps we should move that to "internals" doc and rework this section of the guide... https://issues.apache.org/jira/browse/ZOOKEEPER-220 Patrick -Original Message- From: "Patrick Hunt" <[EMAIL PROTECTED]> Sent: Wednesday, November 12, 2008 2:11pm To: zookeeper-user@hadoop.apache.org Subject: Re: Exists Watch Triggered by Delete Hi Stu, The zk server maintains 2 lists of watches, data and child watches: http://hadoop.apache.org/zookeeper/docs/r3.0.0/zookeeperProgrammers.html#ch_zkWatches (after reviewing this doc I've entered a jira to clarify that the server is maintaining 2 lists being referenced). From the server perspective if you register a watch on a node by calling getData & exists, only a single watch, a data watch, is stored by the server. The client is maintaining lists of watches as well. This is essentially to enable the auto watch reset and multi-watcher features added in v3. Take a look at class ExistsWatchRegistration, it will register client side dataWatches for exists calls -- unless the result code is not 0 (ie NONODE), in which case it will register using existsWatches (again, client side). The comment you referenced in your original email is true - that code should never execute as the existsWatches list only contains watches for NONODE watch registrations (which obv couldn't be deleted since it doesn't exist). Hope this helps, Patrick Stu Hood wrote: I'm running the 3.0.0 release, and I'm receiving a warning thrown by this block of code: case NodeDeleted: synchronized (dataWatches) { addTo(dataWatches.remove(path), result); } // XXX This shouldn't be needed, but just in case synchronized (existWatches) { addTo(existWatches.remove(path), result); LOG.warn("We are triggering an exists watch for delete! Shouldn't happen!"); } synchronized (childWatches) { addTo(childWatches.remove(path), result); } break; -- Why shouldn't an exists watch be triggered by a node being deleted? That is a really common use case in my code, so I want to rule it out as the cause of a bug I'm hunting for. Thanks, Stu Hood Architecture Software Developer Mailtrust, a Division of Rackspace
Re: Exists Watch Triggered by Delete
Thanks a bunch Patrick! I really appreciate your work on ZooKeeper... you have made a tremendous impact on the project. I will try the ZOOKEEPER-221 patch. Thanks, Stu -Original Message- From: "Patrick Hunt" <[EMAIL PROTECTED]> Sent: Wednesday, November 12, 2008 6:37pm To: zookeeper-user@hadoop.apache.org Subject: Re: Exists Watch Triggered by Delete Stu Hood wrote: >> The comment you referenced in your original email is true - that >> code should never execute as the existsWatches list only contains >> watches for NONODE watch registrations (which obv couldn't be >> deleted since it doesn't exist). > So I am experiencing a bug then? Wow, this is a dumb mistake. The log message you pointed to is being output _every time_ this code is called. There needs to be a conditional on the log message to check the result of the remove call. I have entered a JIRA and have already submitted a patch: https://issues.apache.org/jira/browse/ZOOKEEPER-221 So unfortunately we can't tell if this is the source of the issue you are seeing. Can you run with the patch attached to ZOOKEEPER-221? > It may be better not to let the programmer know about those two > lists, and to see if the abstraction can be improved instead. > Sometimes I feel that I have to know too much about the internal > working of ZooKeeper to use its API. That's a good point. We're working to improve the docs & code, I've made a note of it. Perhaps we should move that to "internals" doc and rework this section of the guide... https://issues.apache.org/jira/browse/ZOOKEEPER-220 Patrick > > > -Original Message- From: "Patrick Hunt" <[EMAIL PROTECTED]> > Sent: Wednesday, November 12, 2008 2:11pm To: > zookeeper-user@hadoop.apache.org Subject: Re: Exists Watch Triggered > by Delete > > Hi Stu, > > The zk server maintains 2 lists of watches, data and child watches: > http://hadoop.apache.org/zookeeper/docs/r3.0.0/zookeeperProgrammers.html#ch_zkWatches > (after reviewing this doc I've entered a jira to clarify that the > server is maintaining 2 lists being referenced). From the server > perspective if you register a watch on a node by calling getData & > exists, only a single watch, a data watch, is stored by the server. > > The client is maintaining lists of watches as well. This is > essentially to enable the auto watch reset and multi-watcher features > added in v3. > > Take a look at class ExistsWatchRegistration, it will register client > side dataWatches for exists calls -- unless the result code is not 0 > (ie NONODE), in which case it will register using existsWatches > (again, client side). > > The comment you referenced in your original email is true - that code > should never execute as the existsWatches list only contains watches > for NONODE watch registrations (which obv couldn't be deleted since > it doesn't exist). > > Hope this helps, > > Patrick > > Stu Hood wrote: >> I'm running the 3.0.0 release, and I'm receiving a warning thrown >> by this block of code: >>> case NodeDeleted: synchronized (dataWatches) { >>> addTo(dataWatches.remove(path), result); } // XXX This shouldn't >>> be needed, but just in case synchronized (existWatches) { >>> addTo(existWatches.remove(path), result); LOG.warn("We are >>> triggering an exists watch for delete! Shouldn't happen!"); } >>> synchronized (childWatches) { addTo(childWatches.remove(path), >>> result); } break; >> -- >> >> Why shouldn't an exists watch be triggered by a node being deleted? >> That is a really common use case in my code, so I want to rule it >> out as the cause of a bug I'm hunting for. >> >> Thanks, >> >> Stu Hood Architecture Software Developer Mailtrust, a Division of >> Rackspace >> > >
Re: Exists Watch Triggered by Delete
Stu Hood wrote: The comment you referenced in your original email is true - that code should never execute as the existsWatches list only contains watches for NONODE watch registrations (which obv couldn't be deleted since it doesn't exist). So I am experiencing a bug then? Wow, this is a dumb mistake. The log message you pointed to is being output _every time_ this code is called. There needs to be a conditional on the log message to check the result of the remove call. I have entered a JIRA and have already submitted a patch: https://issues.apache.org/jira/browse/ZOOKEEPER-221 So unfortunately we can't tell if this is the source of the issue you are seeing. Can you run with the patch attached to ZOOKEEPER-221? It may be better not to let the programmer know about those two lists, and to see if the abstraction can be improved instead. Sometimes I feel that I have to know too much about the internal working of ZooKeeper to use its API. That's a good point. We're working to improve the docs & code, I've made a note of it. Perhaps we should move that to "internals" doc and rework this section of the guide... https://issues.apache.org/jira/browse/ZOOKEEPER-220 Patrick -Original Message- From: "Patrick Hunt" <[EMAIL PROTECTED]> Sent: Wednesday, November 12, 2008 2:11pm To: zookeeper-user@hadoop.apache.org Subject: Re: Exists Watch Triggered by Delete Hi Stu, The zk server maintains 2 lists of watches, data and child watches: http://hadoop.apache.org/zookeeper/docs/r3.0.0/zookeeperProgrammers.html#ch_zkWatches (after reviewing this doc I've entered a jira to clarify that the server is maintaining 2 lists being referenced). From the server perspective if you register a watch on a node by calling getData & exists, only a single watch, a data watch, is stored by the server. The client is maintaining lists of watches as well. This is essentially to enable the auto watch reset and multi-watcher features added in v3. Take a look at class ExistsWatchRegistration, it will register client side dataWatches for exists calls -- unless the result code is not 0 (ie NONODE), in which case it will register using existsWatches (again, client side). The comment you referenced in your original email is true - that code should never execute as the existsWatches list only contains watches for NONODE watch registrations (which obv couldn't be deleted since it doesn't exist). Hope this helps, Patrick Stu Hood wrote: I'm running the 3.0.0 release, and I'm receiving a warning thrown by this block of code: case NodeDeleted: synchronized (dataWatches) { addTo(dataWatches.remove(path), result); } // XXX This shouldn't be needed, but just in case synchronized (existWatches) { addTo(existWatches.remove(path), result); LOG.warn("We are triggering an exists watch for delete! Shouldn't happen!"); } synchronized (childWatches) { addTo(childWatches.remove(path), result); } break; -- Why shouldn't an exists watch be triggered by a node being deleted? That is a really common use case in my code, so I want to rule it out as the cause of a bug I'm hunting for. Thanks, Stu Hood Architecture Software Developer Mailtrust, a Division of Rackspace
Re: Exists Watch Triggered by Delete
> The comment you referenced in your original email is true - that code > should never execute as the existsWatches list only contains watches for > NONODE watch registrations (which obv couldn't be deleted since it > doesn't exist). So I am experiencing a bug then? It may be better not to let the programmer know about those two lists, and to see if the abstraction can be improved instead. Sometimes I feel that I have to know too much about the internal working of ZooKeeper to use its API. Thanks Patrick, Stu -Original Message- From: "Patrick Hunt" <[EMAIL PROTECTED]> Sent: Wednesday, November 12, 2008 2:11pm To: zookeeper-user@hadoop.apache.org Subject: Re: Exists Watch Triggered by Delete Hi Stu, The zk server maintains 2 lists of watches, data and child watches: http://hadoop.apache.org/zookeeper/docs/r3.0.0/zookeeperProgrammers.html#ch_zkWatches (after reviewing this doc I've entered a jira to clarify that the server is maintaining 2 lists being referenced). From the server perspective if you register a watch on a node by calling getData & exists, only a single watch, a data watch, is stored by the server. The client is maintaining lists of watches as well. This is essentially to enable the auto watch reset and multi-watcher features added in v3. Take a look at class ExistsWatchRegistration, it will register client side dataWatches for exists calls -- unless the result code is not 0 (ie NONODE), in which case it will register using existsWatches (again, client side). The comment you referenced in your original email is true - that code should never execute as the existsWatches list only contains watches for NONODE watch registrations (which obv couldn't be deleted since it doesn't exist). Hope this helps, Patrick Stu Hood wrote: > I'm running the 3.0.0 release, and I'm receiving a warning thrown by this > block of code: >> case NodeDeleted: >> synchronized (dataWatches) { >> addTo(dataWatches.remove(path), result); >> } >> // XXX This shouldn't be needed, but just in case >> synchronized (existWatches) { >> addTo(existWatches.remove(path), result); >> LOG.warn("We are triggering an exists watch for delete! Shouldn't >> happen!"); >> } >> synchronized (childWatches) { >> addTo(childWatches.remove(path), result); >> } >> break; > > -- > > Why shouldn't an exists watch be triggered by a node being deleted? That is a > really common use case in my code, so I want to rule it out as the cause of a > bug I'm hunting for. > > Thanks, > > Stu Hood > Architecture Software Developer > Mailtrust, a Division of Rackspace >
Re: Exists Watch Triggered by Delete
Hi Stu, The zk server maintains 2 lists of watches, data and child watches: http://hadoop.apache.org/zookeeper/docs/r3.0.0/zookeeperProgrammers.html#ch_zkWatches (after reviewing this doc I've entered a jira to clarify that the server is maintaining 2 lists being referenced). From the server perspective if you register a watch on a node by calling getData & exists, only a single watch, a data watch, is stored by the server. The client is maintaining lists of watches as well. This is essentially to enable the auto watch reset and multi-watcher features added in v3. Take a look at class ExistsWatchRegistration, it will register client side dataWatches for exists calls -- unless the result code is not 0 (ie NONODE), in which case it will register using existsWatches (again, client side). The comment you referenced in your original email is true - that code should never execute as the existsWatches list only contains watches for NONODE watch registrations (which obv couldn't be deleted since it doesn't exist). Hope this helps, Patrick Stu Hood wrote: I'm running the 3.0.0 release, and I'm receiving a warning thrown by this block of code: case NodeDeleted: synchronized (dataWatches) { addTo(dataWatches.remove(path), result); } // XXX This shouldn't be needed, but just in case synchronized (existWatches) { addTo(existWatches.remove(path), result); LOG.warn("We are triggering an exists watch for delete! Shouldn't happen!"); } synchronized (childWatches) { addTo(childWatches.remove(path), result); } break; -- Why shouldn't an exists watch be triggered by a node being deleted? That is a really common use case in my code, so I want to rule it out as the cause of a bug I'm hunting for. Thanks, Stu Hood Architecture Software Developer Mailtrust, a Division of Rackspace