[jira] Commented: (ZOOKEEPER-702) GSoC 2010: Failure Detector Model
[ https://issues.apache.org/jira/browse/ZOOKEEPER-702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900197#action_12900197 ] Flavio Junqueira commented on ZOOKEEPER-702: Thanks for the comments, Ivan. I particularly agree with your point on building a list every time. I was actually wondering if there is any other important performance penalty we might be introducing. I can't think of any at the moment, but it be good to keep it in mind. The use of hbReceived and appMsgReceived does not bother me because they make more explicit what they do, but I agree it is more flexible the way you propose. Abmar, I went back to the papers and contrasted your implementation against the proposed algorithms. The only issue I could find is that the Bertier approach computes EA differently when j n. I suggest we follow their description. The others seem fine to me. GSoC 2010: Failure Detector Model - Key: ZOOKEEPER-702 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-702 Project: Zookeeper Issue Type: Wish Reporter: Henry Robinson Assignee: Abmar Barros Attachments: bertier-pseudo.txt, bertier-pseudo.txt, chen-pseudo.txt, chen-pseudo.txt, phiaccrual-pseudo.txt, phiaccrual-pseudo.txt, ZOOKEEPER-702-code.patch, ZOOKEEPER-702-doc.patch, ZOOKEEPER-702.patch, ZOOKEEPER-702.patch, ZOOKEEPER-702.patch, ZOOKEEPER-702.patch, ZOOKEEPER-702.patch, ZOOKEEPER-702.patch, ZOOKEEPER-702.patch, ZOOKEEPER-702.patch, ZOOKEEPER-702.patch, ZOOKEEPER-702.patch Failure Detector Module Possible Mentor Henry Robinson (henry at apache dot org) Requirements Java, some distributed systems knowledge, comfort implementing distributed systems protocols Description ZooKeeper servers detects the failure of other servers and clients by counting the number of 'ticks' for which it doesn't get a heartbeat from other machines. This is the 'timeout' method of failure detection and works very well; however it is possible that it is too aggressive and not easily tuned for some more unusual ZooKeeper installations (such as in a wide-area network, or even in a mobile ad-hoc network). This project would abstract the notion of failure detection to a dedicated Java module, and implement several failure detectors to compare and contrast their appropriateness for ZooKeeper. For example, Apache Cassandra uses a phi-accrual failure detector (http://ddsg.jaist.ac.jp/pub/HDY+04.pdf) which is much more tunable and has some very interesting properties. This is a great project if you are interested in distributed algorithms, or want to help re-factor some of ZooKeeper's internal code. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Build failed in Hudson: ZooKeeper-trunk #909
See https://hudson.apache.org/hudson/job/ZooKeeper-trunk/909/ -- [...truncated 510 lines...] [javadoc] Javadoc execution [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/AsyncCallback.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/CreateMode.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/KeeperException.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/ServerAdminClient.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/Watcher.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/ZooDefs.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/ZooKeeper.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/ZooKeeperMain.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/server/LogFormatter.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/server/PurgeTxnLog.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/server/ZooKeeperServerMain.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeerMain.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/server/upgrade/UpgradeMain.java... [javadoc] Loading source files for package org.apache.zookeeper.data... [javadoc] Constructing Javadoc information... [javadoc] Standard Doclet version 1.6.0_11 [javadoc] Building tree for all the packages and classes... [javadoc] Building index for all the packages and classes... [javadoc] Building index for all classes... javadoc-jar: [jar] Building jar: https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/build/zookeeper-3.4.0-javadoc.jar ivy-retrieve-jdiff: [mkdir] Created dir: https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/build/jdiff/lib [ivy:retrieve] :: resolving dependencies :: org.apache.zookeeper#zookeeper;3.4.0 [ivy:retrieve] confs: [jdiff] [ivy:retrieve] found jdiff#jdiff;1.0.9 in default [ivy:retrieve] found xerces#xerces;1.4.4 in default [ivy:retrieve] :: resolution report :: resolve 111ms :: artifacts dl 4ms - | |modules|| artifacts | | conf | number| search|dwnlded|evicted|| number|dwnlded| - | jdiff | 2 | 0 | 0 | 0 || 2 | 0 | - [ivy:retrieve] :: retrieving :: org.apache.zookeeper#zookeeper [ivy:retrieve] confs: [jdiff] [ivy:retrieve] 2 artifacts copied, 0 already retrieved (1896kB/12ms) write-null: api-xml: [javadoc] Generating Javadoc [javadoc] Javadoc execution [javadoc] Loading source files for package org.apache.zookeeper... [javadoc] Constructing Javadoc information... [javadoc] JDiff: doclet started ... [javadoc] JDiff: writing the API to file 'https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/lib/jdiff/zookeeper_3.4.0.xml'... [javadoc] https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/ClientWatchManager.java:38: warning - @return tag has no arguments. [javadoc] JDiff: finished (took 0s, not including scanning the source files). [javadoc] 1 warning api-report: [mkdir] Created dir: https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/build/docs/jdiff [javadoc] Generating Javadoc [javadoc] Javadoc execution [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/build/jdiff/lib/Null.java... [javadoc] Loading source files for package org.apache.jute.compiler... [javadoc] Loading source files for package org.apache.jute.compiler.generated... [javadoc] Loading source files for package org.apache.zookeeper... [javadoc] Loading source files for package org.apache.zookeeper.common... [javadoc] Loading
[jira] Created: (ZOOKEEPER-853) Make zookeeper.is_unrecoverable return True or False and not an integer
Make zookeeper.is_unrecoverable return True or False and not an integer
---
Key: ZOOKEEPER-853
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-853
Project: Zookeeper
Issue Type: Improvement
Components: contrib-bindings
Reporter: Andrei Savu
Assignee: Andrei Savu
Priority: Minor
Fix For: 3.4.0
This is a patch that fixes a TODO from the python zookeeper extension, it makes
{{zookeeper.is_unrecoverable}} return {{True}} or {{False}} and not an integer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-853) Make zookeeper.is_unrecoverable return True or False and not an integer
[
https://issues.apache.org/jira/browse/ZOOKEEPER-853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Savu updated ZOOKEEPER-853:
--
Attachment: ZOOKEEPER-853.patch
Make zookeeper.is_unrecoverable return True or False and not an integer
---
Key: ZOOKEEPER-853
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-853
Project: Zookeeper
Issue Type: Improvement
Components: contrib-bindings
Reporter: Andrei Savu
Assignee: Andrei Savu
Priority: Minor
Fix For: 3.4.0
Attachments: ZOOKEEPER-853.patch
This is a patch that fixes a TODO from the python zookeeper extension, it
makes {{zookeeper.is_unrecoverable}} return {{True}} or {{False}} and not an
integer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-853) Make zookeeper.is_unrecoverable return True or False and not an integer
[
https://issues.apache.org/jira/browse/ZOOKEEPER-853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Savu updated ZOOKEEPER-853:
--
Status: Patch Available (was: Open)
Make zookeeper.is_unrecoverable return True or False and not an integer
---
Key: ZOOKEEPER-853
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-853
Project: Zookeeper
Issue Type: Improvement
Components: contrib-bindings
Reporter: Andrei Savu
Assignee: Andrei Savu
Priority: Minor
Fix For: 3.4.0
Attachments: ZOOKEEPER-853.patch
This is a patch that fixes a TODO from the python zookeeper extension, it
makes {{zookeeper.is_unrecoverable}} return {{True}} or {{False}} and not an
integer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-853) Make zookeeper.is_unrecoverable return True or False and not an integer
[
https://issues.apache.org/jira/browse/ZOOKEEPER-853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Savu updated ZOOKEEPER-853:
--
Status: Open (was: Patch Available)
Make zookeeper.is_unrecoverable return True or False and not an integer
---
Key: ZOOKEEPER-853
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-853
Project: Zookeeper
Issue Type: Improvement
Components: contrib-bindings
Reporter: Andrei Savu
Assignee: Andrei Savu
Priority: Minor
Fix For: 3.4.0
Attachments: ZOOKEEPER-853.patch
This is a patch that fixes a TODO from the python zookeeper extension, it
makes {{zookeeper.is_unrecoverable}} return {{True}} or {{False}} and not an
integer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-853) Make zookeeper.is_unrecoverable return True or False and not an integer
[
https://issues.apache.org/jira/browse/ZOOKEEPER-853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Savu updated ZOOKEEPER-853:
--
Status: Patch Available (was: Open)
Make zookeeper.is_unrecoverable return True or False and not an integer
---
Key: ZOOKEEPER-853
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-853
Project: Zookeeper
Issue Type: Improvement
Components: contrib-bindings
Reporter: Andrei Savu
Assignee: Andrei Savu
Priority: Minor
Fix For: 3.4.0
Attachments: ZOOKEEPER-853.patch, ZOOKEEPER-853.patch
This is a patch that fixes a TODO from the python zookeeper extension, it
makes {{zookeeper.is_unrecoverable}} return {{True}} or {{False}} and not an
integer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-853) Make zookeeper.is_unrecoverable return True or False and not an integer
[
https://issues.apache.org/jira/browse/ZOOKEEPER-853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Savu updated ZOOKEEPER-853:
--
Attachment: ZOOKEEPER-853.patch
Fixed patch to do proper reference counting when returning boolean values.
Make zookeeper.is_unrecoverable return True or False and not an integer
---
Key: ZOOKEEPER-853
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-853
Project: Zookeeper
Issue Type: Improvement
Components: contrib-bindings
Reporter: Andrei Savu
Assignee: Andrei Savu
Priority: Minor
Fix For: 3.4.0
Attachments: ZOOKEEPER-853.patch, ZOOKEEPER-853.patch
This is a patch that fixes a TODO from the python zookeeper extension, it
makes {{zookeeper.is_unrecoverable}} return {{True}} or {{False}} and not an
integer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
handling auth failure in java client
Hi all, I filed this ticket last week: https://issues.apache.org/jira/browse/ZOOKEEPER-844 Currently, the Java client ignores auth failures which is extremely problematic for the deployment I am preparing. I have written a patch to correct the problem by adding an AuthFailed KeeperState and checking the auth responses for the AUTHFAILED error code (patch is now attached to the ticket). I checked the flow vs the c client and it seems to basically match. Is there anything I should be aware of beyond this simple fix? All the testing I've done seems fine. Thanks, Camille
[jira] Commented: (ZOOKEEPER-792) zkpython memory leak
[ https://issues.apache.org/jira/browse/ZOOKEEPER-792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900380#action_12900380 ] Henry Robinson commented on ZOOKEEPER-792: -- Aha - I think I have found the problem, and it was related to this patch. PyObject *ret = Py_BuildValue( (s#,N), buffer,buffer_len, stat_dict ); + free_pywatcher(pw); free(buffer); We shouldn't free the pywatcher_t object here because it may be called later. This was what was causing the segfault I was seeing. I'll upload a new patch with this line removed; I hope it will still fix your memory consumption issues. zkpython memory leak Key: ZOOKEEPER-792 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-792 Project: Zookeeper Issue Type: Bug Components: contrib-bindings Affects Versions: 3.3.1 Environment: vmware workstation - guest OS:Linux python:2.4.3 Reporter: Lei Zhang Assignee: Lei Zhang Fix For: 3.3.2, 3.4.0 Attachments: ZOOKEEPER-792.patch We recently upgraded zookeeper from 3.2.1 to 3.3.1, now we are seeing less client deadlock on session expiration, which is a definite plus! Unfortunately we are seeing memory leak that requires our zk clients to be restarted every half-day. Valgrind result: ==8804== 25 (12 direct, 13 indirect) bytes in 1 blocks are definitely lost in loss record 255 of 670 ==8804==at 0x4021C42: calloc (vg_replace_malloc.c:418) ==8804==by 0x5047B42: parse_acls (zookeeper.c:369) ==8804==by 0x5047EF6: pyzoo_create (zookeeper.c:1009) ==8804==by 0x40786CC: PyCFunction_Call (in /usr/lib/libpython2.4.so.1.0) ==8804==by 0x40B31DC: PyEval_EvalFrame (in /usr/lib/libpython2.4.so.1.0) ==8804==by 0x40B4485: PyEval_EvalCodeEx (in /usr/lib/libpython2.4.so.1.0) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-844) handle auth failure in java client
[
https://issues.apache.org/jira/browse/ZOOKEEPER-844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Hunt updated ZOOKEEPER-844:
---
Status: Patch Available (was: Open)
Assignee: Camille Fournier
Fix Version/s: 3.4.0
handle auth failure in java client
--
Key: ZOOKEEPER-844
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-844
Project: Zookeeper
Issue Type: Improvement
Components: java client
Affects Versions: 3.3.1
Reporter: Camille Fournier
Assignee: Camille Fournier
Fix For: 3.4.0
Attachments: ZOOKEEPER-844.patch
ClientCnxn.java currently has the following code:
if (replyHdr.getXid() == -4) {
// -2 is the xid for AuthPacket
// TODO: process AuthPacket here
if (LOG.isDebugEnabled()) {
LOG.debug(Got auth sessionid:0x
+ Long.toHexString(sessionId));
}
return;
}
Auth failures appear to cause the server to disconnect but the client never
gets a proper state change or notification that auth has failed, which makes
handling this scenario very difficult as it causes the client to go into a
loop of sending bad auth, getting disconnected, trying to reconnect, sending
bad auth again, over and over.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: handling auth failure in java client
Hi Camille, thanks for the submission! I've marked the jira as patch available (by clicking the submit link), someone will take a look (review). I see that you wrote a test as well, which is great. Thanks, Patrick On 08/19/2010 09:34 AM, Fournier, Camille F. [Tech] wrote: Hi all, I filed this ticket last week: https://issues.apache.org/jira/browse/ZOOKEEPER-844 Currently, the Java client ignores auth failures which is extremely problematic for the deployment I am preparing. I have written a patch to correct the problem by adding an AuthFailed KeeperState and checking the auth responses for the AUTHFAILED error code (patch is now attached to the ticket). I checked the flow vs the c client and it seems to basically match. Is there anything I should be aware of beyond this simple fix? All the testing I've done seems fine. Thanks, Camille
[jira] Updated: (ZOOKEEPER-792) zkpython memory leak
[ https://issues.apache.org/jira/browse/ZOOKEEPER-792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henry Robinson updated ZOOKEEPER-792: - Attachment: ZOOKEEPER-792.patch I forgot --no-prefix. Plus ca change, plus c'est la meme chose. zkpython memory leak Key: ZOOKEEPER-792 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-792 Project: Zookeeper Issue Type: Bug Components: contrib-bindings Affects Versions: 3.3.1 Environment: vmware workstation - guest OS:Linux python:2.4.3 Reporter: Lei Zhang Assignee: Lei Zhang Fix For: 3.3.2, 3.4.0 Attachments: ZOOKEEPER-792.patch, ZOOKEEPER-792.patch, ZOOKEEPER-792.patch We recently upgraded zookeeper from 3.2.1 to 3.3.1, now we are seeing less client deadlock on session expiration, which is a definite plus! Unfortunately we are seeing memory leak that requires our zk clients to be restarted every half-day. Valgrind result: ==8804== 25 (12 direct, 13 indirect) bytes in 1 blocks are definitely lost in loss record 255 of 670 ==8804==at 0x4021C42: calloc (vg_replace_malloc.c:418) ==8804==by 0x5047B42: parse_acls (zookeeper.c:369) ==8804==by 0x5047EF6: pyzoo_create (zookeeper.c:1009) ==8804==by 0x40786CC: PyCFunction_Call (in /usr/lib/libpython2.4.so.1.0) ==8804==by 0x40B31DC: PyEval_EvalFrame (in /usr/lib/libpython2.4.so.1.0) ==8804==by 0x40B4485: PyEval_EvalCodeEx (in /usr/lib/libpython2.4.so.1.0) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
BookKeeper build failing.
Seems bk is dependent on some zk code, which I broke with my latest commit for ZOOKEEPER-733 (netty) Flavio, can you handle addressing this or should I? Patrick Original Message Subject: Build failed in Hudson: ZooKeeper-trunk #909 Date: Thu, 19 Aug 2010 10:32:18 + (UTC) From: Apache Hudson Server hud...@hudson.apache.org Reply-To: zookeeper-dev@hadoop.apache.org To: zookeeper-dev@hadoop.apache.org See https://hudson.apache.org/hudson/job/ZooKeeper-trunk/909/ -- [...truncated 510 lines...] [javadoc] Javadoc execution [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/AsyncCallback.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/CreateMode.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/KeeperException.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/ServerAdminClient.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/Watcher.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/ZooDefs.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/ZooKeeper.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/ZooKeeperMain.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/server/LogFormatter.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/server/PurgeTxnLog.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/server/ZooKeeperServerMain.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeerMain.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/server/upgrade/UpgradeMain.java... [javadoc] Loading source files for package org.apache.zookeeper.data... [javadoc] Constructing Javadoc information... [javadoc] Standard Doclet version 1.6.0_11 [javadoc] Building tree for all the packages and classes... [javadoc] Building index for all the packages and classes... [javadoc] Building index for all classes... javadoc-jar: [jar] Building jar: https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/build/zookeeper-3.4.0-javadoc.jar ivy-retrieve-jdiff: [mkdir] Created dir: https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/build/jdiff/lib [ivy:retrieve] :: resolving dependencies :: org.apache.zookeeper#zookeeper;3.4.0 [ivy:retrieve] confs: [jdiff] [ivy:retrieve] found jdiff#jdiff;1.0.9 in default [ivy:retrieve] found xerces#xerces;1.4.4 in default [ivy:retrieve] :: resolution report :: resolve 111ms :: artifacts dl 4ms - | |modules|| artifacts | | conf | number| search|dwnlded|evicted|| number|dwnlded| - | jdiff | 2 | 0 | 0 | 0 || 2 | 0 | - [ivy:retrieve] :: retrieving :: org.apache.zookeeper#zookeeper [ivy:retrieve] confs: [jdiff] [ivy:retrieve] 2 artifacts copied, 0 already retrieved (1896kB/12ms) write-null: api-xml: [javadoc] Generating Javadoc [javadoc] Javadoc execution [javadoc] Loading source files for package org.apache.zookeeper... [javadoc] Constructing Javadoc information... [javadoc] JDiff: doclet started ... [javadoc] JDiff: writing the API to file 'https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/lib/jdiff/zookeeper_3.4.0.xml'... [javadoc] https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/ClientWatchManager.java:38: warning - @return tag has no arguments. [javadoc] JDiff: finished (took 0s, not including scanning the source files). [javadoc] 1 warning api-report: [mkdir] Created dir: https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/build/docs/jdiff [javadoc] Generating Javadoc [javadoc] Javadoc execution [javadoc]
[jira] Commented: (ZOOKEEPER-822) Leader election taking a long time to complete
[ https://issues.apache.org/jira/browse/ZOOKEEPER-822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900401#action_12900401 ] Vishal K commented on ZOOKEEPER-822: Hi Flavio, Ah! my trunk is quite old then. But I don't think it is necessary to run with the latest code for debugging this issue. I have identified one problem in WorkerSender.process(). This function calls manager.toSend() whicih calls connectOne. connectOne does a blocking connect (which takes order of minutes to return if a node is down). Thus, WorkerSender.run() will block and not send any successive notifications to other nodes. Let met know what you think I tired with adding timeouts to connectOne, but I am running into similar issue somewhere else. So that didnt fix the problem Leader election taking a long time to complete --- Key: ZOOKEEPER-822 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-822 Project: Zookeeper Issue Type: Bug Components: quorum Affects Versions: 3.3.0 Reporter: Vishal K Priority: Blocker Attachments: 822.tar.gz, rhel.tar.gz, test_zookeeper_1.log, test_zookeeper_2.log, zk_leader_election.tar.gz Created a 3 node cluster. 1 Fail the ZK leader 2. Let leader election finish. Restart the leader and let it join the 3. Repeat After a few rounds leader election takes anywhere 25- 60 seconds to finish. Note- we didn't have any ZK clients and no new znodes were created. zoo.cfg is shown below: #Mon Jul 19 12:15:10 UTC 2010 server.1=192.168.4.12\:2888\:3888 server.0=192.168.4.11\:2888\:3888 clientPort=2181 dataDir=/var/zookeeper syncLimit=2 server.2=192.168.4.13\:2888\:3888 initLimit=5 tickTime=2000 I have attached logs from two nodes that took a long time to form the cluster after failing the leader. The leader was down anyways so logs from that node shouldn't matter. Look for START HERE. Logs after that point should be of our interest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-854) BookKeeper does not compile due to changes in the ZooKeeper code
BookKeeper does not compile due to changes in the ZooKeeper code Key: ZOOKEEPER-854 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-854 Project: Zookeeper Issue Type: Bug Components: contrib-bookkeeper Affects Versions: 3.3.1 Reporter: Flavio Junqueira Fix For: 3.4.0 BookKeeper does not compile due to changes in the NIOServerCnxn class of ZooKeeper. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-854) BookKeeper does not compile due to changes in the ZooKeeper code
[ https://issues.apache.org/jira/browse/ZOOKEEPER-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Junqueira updated ZOOKEEPER-854: --- Attachment: ZOOKEEPER-854.patch Minor changes to LocalBookKeeper to reflect the changes in NIOServerCnxn and NIOServerCnxnFactory. BookKeeper does not compile due to changes in the ZooKeeper code Key: ZOOKEEPER-854 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-854 Project: Zookeeper Issue Type: Bug Components: contrib-bookkeeper Affects Versions: 3.3.1 Reporter: Flavio Junqueira Fix For: 3.4.0 Attachments: ZOOKEEPER-854.patch BookKeeper does not compile due to changes in the NIOServerCnxn class of ZooKeeper. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (ZOOKEEPER-854) BookKeeper does not compile due to changes in the ZooKeeper code
[ https://issues.apache.org/jira/browse/ZOOKEEPER-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Junqueira reassigned ZOOKEEPER-854: -- Assignee: Flavio Junqueira BookKeeper does not compile due to changes in the ZooKeeper code Key: ZOOKEEPER-854 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-854 Project: Zookeeper Issue Type: Bug Components: contrib-bookkeeper Affects Versions: 3.3.1 Reporter: Flavio Junqueira Assignee: Flavio Junqueira Fix For: 3.4.0 Attachments: ZOOKEEPER-854.patch, ZOOKEEPER-854.patch BookKeeper does not compile due to changes in the NIOServerCnxn class of ZooKeeper. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-854) BookKeeper does not compile due to changes in the ZooKeeper code
[ https://issues.apache.org/jira/browse/ZOOKEEPER-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Junqueira updated ZOOKEEPER-854: --- Attachment: ZOOKEEPER-854.patch BookKeeper does not compile due to changes in the ZooKeeper code Key: ZOOKEEPER-854 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-854 Project: Zookeeper Issue Type: Bug Components: contrib-bookkeeper Affects Versions: 3.3.1 Reporter: Flavio Junqueira Fix For: 3.4.0 Attachments: ZOOKEEPER-854.patch, ZOOKEEPER-854.patch BookKeeper does not compile due to changes in the NIOServerCnxn class of ZooKeeper. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-854) BookKeeper does not compile due to changes in the ZooKeeper code
[ https://issues.apache.org/jira/browse/ZOOKEEPER-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Junqueira updated ZOOKEEPER-854: --- Status: Patch Available (was: Open) BookKeeper does not compile due to changes in the ZooKeeper code Key: ZOOKEEPER-854 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-854 Project: Zookeeper Issue Type: Bug Components: contrib-bookkeeper Affects Versions: 3.3.1 Reporter: Flavio Junqueira Assignee: Flavio Junqueira Fix For: 3.4.0 Attachments: ZOOKEEPER-854.patch, ZOOKEEPER-854.patch BookKeeper does not compile due to changes in the NIOServerCnxn class of ZooKeeper. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-775) A large scale pub/sub system
[ https://issues.apache.org/jira/browse/ZOOKEEPER-775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated ZOOKEEPER-775: Attachment: ZOOKEEPER-775.patch this patch make the following changes to the sumitted patch. - removes ltmain.sh - removes client/src/main/cpp/m4/ax_jni_include_dir.m4 - remove author names A large scale pub/sub system Key: ZOOKEEPER-775 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-775 Project: Zookeeper Issue Type: New Feature Components: contrib Reporter: Benjamin Reed Assignee: Benjamin Reed Fix For: 3.4.0 Attachments: libs.zip, libs_2.zip, ZOOKEEPER-775.patch, ZOOKEEPER-775.patch, ZOOKEEPER-775.patch, ZOOKEEPER-775.patch, ZOOKEEPER-775_2.patch, ZOOKEEPER-775_3.patch we have developed a large scale pub/sub system based on ZooKeeper and BookKeeper. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: BookKeeper build failing.
thanks. On 08/19/2010 12:59 PM, Flavio Junqueira wrote: Submitting a patch shortly. -Flavio On Aug 19, 2010, at 8:28 PM, Patrick Hunt wrote: Seems bk is dependent on some zk code, which I broke with my latest commit for ZOOKEEPER-733 (netty) Flavio, can you handle addressing this or should I? Patrick Original Message Subject: Build failed in Hudson: ZooKeeper-trunk #909 Date: Thu, 19 Aug 2010 10:32:18 + (UTC) From: Apache Hudson Server hud...@hudson.apache.org mailto:hud...@hudson.apache.org Reply-To: zookeeper-dev@hadoop.apache.org mailto:zookeeper-dev@hadoop.apache.org To: zookeeper-dev@hadoop.apache.org mailto:zookeeper-dev@hadoop.apache.org See https://hudson.apache.org/hudson/job/ZooKeeper-trunk/909/ -- [...truncated 510 lines...] [javadoc] Javadoc execution [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/AsyncCallback.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/CreateMode.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/KeeperException.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/ServerAdminClient.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/Watcher.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/ZooDefs.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/ZooKeeper.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/ZooKeeperMain.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/server/LogFormatter.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/server/PurgeTxnLog.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/server/ZooKeeperServerMain.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeerMain.java... [javadoc] Loading source file https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/server/upgrade/UpgradeMain.java... [javadoc] Loading source files for package org.apache.zookeeper.data... [javadoc] Constructing Javadoc information... [javadoc] Standard Doclet version 1.6.0_11 [javadoc] Building tree for all the packages and classes... [javadoc] Building index for all the packages and classes... [javadoc] Building index for all classes... javadoc-jar: [jar] Building jar: https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/build/zookeeper-3.4.0-javadoc.jar ivy-retrieve-jdiff: [mkdir] Created dir: https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/build/jdiff/lib [ivy:retrieve] :: resolving dependencies :: org.apache.zookeeper#zookeeper;3.4.0 [ivy:retrieve] confs: [jdiff] [ivy:retrieve] found jdiff#jdiff;1.0.9 in default [ivy:retrieve] found xerces#xerces;1.4.4 in default [ivy:retrieve] :: resolution report :: resolve 111ms :: artifacts dl 4ms - | | modules || artifacts | | conf | number| search|dwnlded|evicted|| number|dwnlded| - | jdiff | 2 | 0 | 0 | 0 || 2 | 0 | - [ivy:retrieve] :: retrieving :: org.apache.zookeeper#zookeeper [ivy:retrieve] confs: [jdiff] [ivy:retrieve] 2 artifacts copied, 0 already retrieved (1896kB/12ms) write-null: api-xml: [javadoc] Generating Javadoc [javadoc] Javadoc execution [javadoc] Loading source files for package org.apache.zookeeper... [javadoc] Constructing Javadoc information... [javadoc] JDiff: doclet started ... [javadoc] JDiff: writing the API to file 'https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/lib/jdiff/zookeeper_3.4.0.xml'... [javadoc] https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/src/java/main/org/apache/zookeeper/ClientWatchManager.java:38: warning - @return tag has no arguments. [javadoc] JDiff: finished (took 0s, not including scanning the source files). [javadoc] 1 warning api-report: [mkdir] Created dir: https://hudson.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/build/docs/jdiff [javadoc] Generating Javadoc [javadoc] Javadoc execution
[jira] Updated: (ZOOKEEPER-775) A large scale pub/sub system
[ https://issues.apache.org/jira/browse/ZOOKEEPER-775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated ZOOKEEPER-775: Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Release Note: A pub sub system using BooKkeeper and ZooKeeper with C++ and Java client bindings. Resolution: Fixed I just committed this. Thanks Ivan, Erwin, Ben. It would be great if you guys can focus on improved documentation for this since it will be critical for adoption of the project. A large scale pub/sub system Key: ZOOKEEPER-775 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-775 Project: Zookeeper Issue Type: New Feature Components: contrib Reporter: Benjamin Reed Assignee: Benjamin Reed Fix For: 3.4.0 Attachments: libs.zip, libs_2.zip, ZOOKEEPER-775.patch, ZOOKEEPER-775.patch, ZOOKEEPER-775.patch, ZOOKEEPER-775.patch, ZOOKEEPER-775_2.patch, ZOOKEEPER-775_3.patch we have developed a large scale pub/sub system based on ZooKeeper and BookKeeper. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-775) A large scale pub/sub system
[ https://issues.apache.org/jira/browse/ZOOKEEPER-775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900507#action_12900507 ] Patrick Hunt commented on ZOOKEEPER-775: Awesome job everyone, really psyched to see this get in! A large scale pub/sub system Key: ZOOKEEPER-775 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-775 Project: Zookeeper Issue Type: New Feature Components: contrib Reporter: Benjamin Reed Assignee: Benjamin Reed Fix For: 3.4.0 Attachments: libs.zip, libs_2.zip, ZOOKEEPER-775.patch, ZOOKEEPER-775.patch, ZOOKEEPER-775.patch, ZOOKEEPER-775.patch, ZOOKEEPER-775_2.patch, ZOOKEEPER-775_3.patch we have developed a large scale pub/sub system based on ZooKeeper and BookKeeper. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (ZOOKEEPER-366) Session timeout detection can go wrong if the leader system time changes
[ https://issues.apache.org/jira/browse/ZOOKEEPER-366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Reed reassigned ZOOKEEPER-366: --- Assignee: Benjamin Reed Session timeout detection can go wrong if the leader system time changes Key: ZOOKEEPER-366 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-366 Project: Zookeeper Issue Type: Bug Reporter: Benjamin Reed Assignee: Benjamin Reed the leader tracks session expirations by calculating when a session will timeout and then periodically checking to see what needs to be timed out based on the current time. this works great as long as the leaders clock progresses at a steady pace. the problem comes when there are big (session size) changes in clock, by ntp for example. if time gets adjusted forward, all the sessions could timeout immediately. if time goes backward sessions that should timeout may take a lot longer to actually expire. this is really just a leader issue. the easiest way to deal with this is to have the leader relinquish leadership if it detects a big jump forward in time. when a new leader gets elected, it will recalculate timeouts of active sessions. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-366) Session timeout detection can go wrong if the leader system time changes
[ https://issues.apache.org/jira/browse/ZOOKEEPER-366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900511#action_12900511 ] Benjamin Reed commented on ZOOKEEPER-366: - after discussion this on the list, we realized that we can detect a big jump in time change in the session expiration thread. since we expire a bucket of sessions each tick, if we run into the situation where we are going to expire more than one bucket in a row, we know we have jumped forward in time. we can smooth the jump by requiring at least a 1/2 ticktime wait between each bucket. Session timeout detection can go wrong if the leader system time changes Key: ZOOKEEPER-366 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-366 Project: Zookeeper Issue Type: Bug Reporter: Benjamin Reed the leader tracks session expirations by calculating when a session will timeout and then periodically checking to see what needs to be timed out based on the current time. this works great as long as the leaders clock progresses at a steady pace. the problem comes when there are big (session size) changes in clock, by ntp for example. if time gets adjusted forward, all the sessions could timeout immediately. if time goes backward sessions that should timeout may take a lot longer to actually expire. this is really just a leader issue. the easiest way to deal with this is to have the leader relinquish leadership if it detects a big jump forward in time. when a new leader gets elected, it will recalculate timeouts of active sessions. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-792) zkpython memory leak
[ https://issues.apache.org/jira/browse/ZOOKEEPER-792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900563#action_12900563 ] Lei Zhang commented on ZOOKEEPER-792: - Thanks a lot. zkpython memory leak Key: ZOOKEEPER-792 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-792 Project: Zookeeper Issue Type: Bug Components: contrib-bindings Affects Versions: 3.3.1 Environment: vmware workstation - guest OS:Linux python:2.4.3 Reporter: Lei Zhang Assignee: Lei Zhang Fix For: 3.3.2, 3.4.0 Attachments: ZOOKEEPER-792.patch, ZOOKEEPER-792.patch, ZOOKEEPER-792.patch We recently upgraded zookeeper from 3.2.1 to 3.3.1, now we are seeing less client deadlock on session expiration, which is a definite plus! Unfortunately we are seeing memory leak that requires our zk clients to be restarted every half-day. Valgrind result: ==8804== 25 (12 direct, 13 indirect) bytes in 1 blocks are definitely lost in loss record 255 of 670 ==8804==at 0x4021C42: calloc (vg_replace_malloc.c:418) ==8804==by 0x5047B42: parse_acls (zookeeper.c:369) ==8804==by 0x5047EF6: pyzoo_create (zookeeper.c:1009) ==8804==by 0x40786CC: PyCFunction_Call (in /usr/lib/libpython2.4.so.1.0) ==8804==by 0x40B31DC: PyEval_EvalFrame (in /usr/lib/libpython2.4.so.1.0) ==8804==by 0x40B4485: PyEval_EvalCodeEx (in /usr/lib/libpython2.4.so.1.0) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
