[jira] [Commented] (ZOOKEEPER-1136) NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the twiki
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13071788#comment-13071788 ] Mahadev konar commented on ZOOKEEPER-1136: -- Ben, are you working on this? > NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the > twiki > --- > > Key: ZOOKEEPER-1136 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1136 > Project: ZooKeeper > Issue Type: Bug >Reporter: Benjamin Reed >Priority: Blocker > Fix For: 3.3.4, 3.4.0 > > > the NEW_LEADER message was sent at the beginning of the sync phase in Zab > pre1.0, but it must be at the end in Zab 1.0. if the protocol is 1.0 or > greater we need to queue rather than send the packet. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1136) NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the twiki
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13071855#comment-13071855 ] Benjamin Reed commented on ZOOKEEPER-1136: -- yes, hoping to have a patch today. > NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the > twiki > --- > > Key: ZOOKEEPER-1136 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1136 > Project: ZooKeeper > Issue Type: Bug >Reporter: Benjamin Reed >Priority: Blocker > Fix For: 3.3.4, 3.4.0 > > > the NEW_LEADER message was sent at the beginning of the sync phase in Zab > pre1.0, but it must be at the end in Zab 1.0. if the protocol is 1.0 or > greater we need to queue rather than send the packet. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1136) NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the twiki
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084214#comment-13084214 ] Hadoop QA commented on ZOOKEEPER-1136: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12490258/ZOOKEEPER-1136.patch against trunk revision 1156845. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/454//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/454//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/454//console This message is automatically generated. > NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the > twiki > --- > > Key: ZOOKEEPER-1136 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1136 > Project: ZooKeeper > Issue Type: Bug >Reporter: Benjamin Reed >Priority: Blocker > Fix For: 3.3.4, 3.4.0 > > Attachments: ZOOKEEPER-1136.patch > > > the NEW_LEADER message was sent at the beginning of the sync phase in Zab > pre1.0, but it must be at the end in Zab 1.0. if the protocol is 1.0 or > greater we need to queue rather than send the packet. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1136) NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the twiki
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084862#comment-13084862 ] Mahadev konar commented on ZOOKEEPER-1136: -- ben, could you please take a look at the failing test? > NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the > twiki > --- > > Key: ZOOKEEPER-1136 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1136 > Project: ZooKeeper > Issue Type: Bug >Reporter: Benjamin Reed >Assignee: Benjamin Reed >Priority: Blocker > Fix For: 3.3.4, 3.4.0 > > Attachments: ZOOKEEPER-1136.patch > > > the NEW_LEADER message was sent at the beginning of the sync phase in Zab > pre1.0, but it must be at the end in Zab 1.0. if the protocol is 1.0 or > greater we need to queue rather than send the packet. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1136) NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the twiki
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087544#comment-13087544 ] Hadoop QA commented on ZOOKEEPER-1136: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12490918/ZOOKEEPER-1136.patch against trunk revision 1159432. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/467//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/467//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/467//console This message is automatically generated. > NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the > twiki > --- > > Key: ZOOKEEPER-1136 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1136 > Project: ZooKeeper > Issue Type: Bug >Reporter: Benjamin Reed >Assignee: Benjamin Reed >Priority: Blocker > Fix For: 3.3.4, 3.4.0 > > Attachments: ZOOKEEPER-1136.patch, ZOOKEEPER-1136.patch > > > the NEW_LEADER message was sent at the beginning of the sync phase in Zab > pre1.0, but it must be at the end in Zab 1.0. if the protocol is 1.0 or > greater we need to queue rather than send the packet. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1136) NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the twiki
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087782#comment-13087782 ] Flavio Junqueira commented on ZOOKEEPER-1136: - Great that you've been able to spot the problem, Ben. A few quick comments: # I don't understand why you're checking if a snapshot has been taken. Is it for backward compatibility? # There are two blocks in LearnerHandler that seem to be doing only reformatting. Do we want to have those changes too? # There is no test, but it sounds like a good idea to have one, no? > NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the > twiki > --- > > Key: ZOOKEEPER-1136 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1136 > Project: ZooKeeper > Issue Type: Bug >Reporter: Benjamin Reed >Assignee: Benjamin Reed >Priority: Blocker > Fix For: 3.3.4, 3.4.0 > > Attachments: ZOOKEEPER-1136.patch, ZOOKEEPER-1136.patch > > > the NEW_LEADER message was sent at the beginning of the sync phase in Zab > pre1.0, but it must be at the end in Zab 1.0. if the protocol is 1.0 or > greater we need to queue rather than send the packet. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1136) NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the twiki
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088295#comment-13088295 ] Benjamin Reed commented on ZOOKEEPER-1136: -- 1. yes, i'll put a comment to that effect in the patch 2. it is only reformatting. the previous patch was checked in with tabs 3. yes, we should add a test. let me see what i can do. > NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the > twiki > --- > > Key: ZOOKEEPER-1136 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1136 > Project: ZooKeeper > Issue Type: Bug >Reporter: Benjamin Reed >Assignee: Benjamin Reed >Priority: Blocker > Fix For: 3.3.4, 3.4.0 > > Attachments: ZOOKEEPER-1136.patch, ZOOKEEPER-1136.patch > > > the NEW_LEADER message was sent at the beginning of the sync phase in Zab > pre1.0, but it must be at the end in Zab 1.0. if the protocol is 1.0 or > greater we need to queue rather than send the packet. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1136) NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the twiki
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093490#comment-13093490 ] Mahadev konar commented on ZOOKEEPER-1136: -- Ben, Any update on this? > NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the > twiki > --- > > Key: ZOOKEEPER-1136 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1136 > Project: ZooKeeper > Issue Type: Bug >Reporter: Benjamin Reed >Assignee: Benjamin Reed >Priority: Blocker > Fix For: 3.3.4, 3.4.0 > > Attachments: ZOOKEEPER-1136.patch, ZOOKEEPER-1136.patch > > > the NEW_LEADER message was sent at the beginning of the sync phase in Zab > pre1.0, but it must be at the end in Zab 1.0. if the protocol is 1.0 or > greater we need to queue rather than send the packet. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1136) NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the twiki
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096814#comment-13096814 ] Hadoop QA commented on ZOOKEEPER-1136: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12492950/ZOOKEEPER-1136.patch against trunk revision 1164758. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/498//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/498//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/498//console This message is automatically generated. > NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the > twiki > --- > > Key: ZOOKEEPER-1136 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1136 > Project: ZooKeeper > Issue Type: Bug >Reporter: Benjamin Reed >Assignee: Benjamin Reed >Priority: Blocker > Fix For: 3.3.4, 3.4.0 > > Attachments: ZOOKEEPER-1136.patch, ZOOKEEPER-1136.patch, > ZOOKEEPER-1136.patch > > > the NEW_LEADER message was sent at the beginning of the sync phase in Zab > pre1.0, but it must be at the end in Zab 1.0. if the protocol is 1.0 or > greater we need to queue rather than send the packet. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1136) NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the twiki
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104289#comment-13104289
]
Mahadev konar commented on ZOOKEEPER-1136:
--
The patch looks good to me, but there are changes that werent necessary in the
patch, like:
{code}
+
sock.setSoTimeout(leader.self.getTickTime()*leader.self.getInitLimit());
{code}
in LearnerHandler.java and also the ByteBufferOutputStream changes.
We should enumerate the list of changes made in a patch so that its consistent
with the changes in the patch.
I will go ahead and commit this for now.
> NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the
> twiki
> ---
>
> Key: ZOOKEEPER-1136
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1136
> Project: ZooKeeper
> Issue Type: Bug
>Reporter: Benjamin Reed
>Assignee: Benjamin Reed
>Priority: Blocker
> Fix For: 3.3.4, 3.4.0
>
> Attachments: ZOOKEEPER-1136.patch, ZOOKEEPER-1136.patch,
> ZOOKEEPER-1136.patch
>
>
> the NEW_LEADER message was sent at the beginning of the sync phase in Zab
> pre1.0, but it must be at the end in Zab 1.0. if the protocol is 1.0 or
> greater we need to queue rather than send the packet.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1136) NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the twiki
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105267#comment-13105267 ] Hudson commented on ZOOKEEPER-1136: --- Integrated in ZooKeeper-trunk #1304 (See [https://builds.apache.org/job/ZooKeeper-trunk/1304/]) ZOOKEEPER-1136. NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the twiki (breed via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1170452 Files : * /zookeeper/trunk/CHANGES.txt * /zookeeper/trunk/src/java/main/org/apache/zookeeper/server/ByteBufferInputStream.java * /zookeeper/trunk/src/java/main/org/apache/zookeeper/server/ByteBufferOutputStream.java * /zookeeper/trunk/src/java/main/org/apache/zookeeper/server/DataTree.java * /zookeeper/trunk/src/java/main/org/apache/zookeeper/server/FinalRequestProcessor.java * /zookeeper/trunk/src/java/main/org/apache/zookeeper/server/PrepRequestProcessor.java * /zookeeper/trunk/src/java/main/org/apache/zookeeper/server/ZooKeeperServer.java * /zookeeper/trunk/src/java/main/org/apache/zookeeper/server/quorum/Leader.java * /zookeeper/trunk/src/java/main/org/apache/zookeeper/server/quorum/Learner.java * /zookeeper/trunk/src/java/main/org/apache/zookeeper/server/quorum/LearnerHandler.java * /zookeeper/trunk/src/java/test/org/apache/zookeeper/server/quorum/Zab1_0Test.java > NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the > twiki > --- > > Key: ZOOKEEPER-1136 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1136 > Project: ZooKeeper > Issue Type: Bug >Reporter: Benjamin Reed >Assignee: Benjamin Reed >Priority: Blocker > Fix For: 3.4.0 > > Attachments: ZOOKEEPER-1136.patch, ZOOKEEPER-1136.patch, > ZOOKEEPER-1136.patch > > > the NEW_LEADER message was sent at the beginning of the sync phase in Zab > pre1.0, but it must be at the end in Zab 1.0. if the protocol is 1.0 or > greater we need to queue rather than send the packet. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1136) NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the twiki
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141234#comment-13141234 ] Camille Fournier commented on ZOOKEEPER-1136: - This change causes a concurrency bug. Specifically: 1. Follower rejoins, gets snap from leader 2. Follower gets NEWLEADER message and takes a snapshot 3. Follower gets some additional tranactions forwarded from leader, applies these directly to data tree 4. Follower gets an UPTODATE message, does not take a snapshot 5. Follower starts following, writes some new transactions to its log, and is killed before it takes another snapshot 6. Follower restarts and gets a DIFF from the leader The transactions that came in between NEWLEADER and UPTODATE are lost because they never go anywhere but the internal data tree, and if that tree isn't snapshotted and the follower restarts with only a DIFF, the follower will lose these transactions. I think the proper thing to do is snapshot after UPTODATE, but I'm not sure why we changed this to snapshot after NEWLEADER instead. The wiki doesn't seem to explain that clearly. If one of you could check on https://issues.apache.org/jira/browse/ZOOKEEPER-1264 and let me know the reasoning, that would be helpful. > NEW_LEADER should be queued not sent to match the Zab 1.0 protocol on the > twiki > --- > > Key: ZOOKEEPER-1136 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1136 > Project: ZooKeeper > Issue Type: Bug >Reporter: Benjamin Reed >Assignee: Benjamin Reed >Priority: Blocker > Fix For: 3.4.0 > > Attachments: ZOOKEEPER-1136.patch, ZOOKEEPER-1136.patch, > ZOOKEEPER-1136.patch > > > the NEW_LEADER message was sent at the beginning of the sync phase in Zab > pre1.0, but it must be at the end in Zab 1.0. if the protocol is 1.0 or > greater we need to queue rather than send the packet. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
