[jira] Commented: (ZOOKEEPER-576) docs need to be updated for session moved exception and how to handle it

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780341#action_12780341
 ] 

Hadoop QA commented on ZOOKEEPER-576:
-

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12425543/ZOOKEEPER-576.patch
  against trunk revision 882313.

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no tests are needed for this patch.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/70/testReport/
Findbugs warnings: 
http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/70/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/70/console

This message is automatically generated.

> docs need to be updated for session moved exception and how to handle it
> 
>
> Key: ZOOKEEPER-576
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-576
> Project: Zookeeper
>  Issue Type: Bug
>Reporter: Mahadev konar
>Assignee: Benjamin Reed
> Fix For: 3.2.2, 3.3.0
>
> Attachments: ZOOKEEPER-576.patch
>
>
> the handling and implications of session moved exception should be documented.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-585) Update README for zkpython in 3.2.2

[Henry Robinson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Henry Robinson updated ZOOKEEPER-585:
-

Attachment: ZOOKEEPER-585.patch

Patch: bumps 'version' to 0.5, details the changes, removes a couple of 
warnings. 

> Update README for zkpython in 3.2.2
> ---
>
> Key: ZOOKEEPER-585
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-585
> Project: Zookeeper
>  Issue Type: Improvement
>  Components: contrib-bindings
>Reporter: Henry Robinson
>Assignee: Henry Robinson
>Priority: Minor
> Fix For: 3.2.2
>
> Attachments: ZOOKEEPER-585.patch
>
>
> zkpython has a few improvements going into 3.2.2, and its README needs a 
> short update to reflect this. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (ZOOKEEPER-585) Update README for zkpython in 3.2.2

[Henry Robinson (JIRA)]

Update README for zkpython in 3.2.2
---

 Key: ZOOKEEPER-585
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-585
 Project: Zookeeper
  Issue Type: Improvement
  Components: contrib-bindings
Reporter: Henry Robinson
Assignee: Henry Robinson
Priority: Minor
 Fix For: 3.2.2
 Attachments: ZOOKEEPER-585.patch

zkpython has a few improvements going into 3.2.2, and its README needs a short 
update to reflect this. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-576) docs need to be updated for session moved exception and how to handle it

[Benjamin Reed (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Reed updated ZOOKEEPER-576:


Attachment: ZOOKEEPER-576.patch

> docs need to be updated for session moved exception and how to handle it
> 
>
> Key: ZOOKEEPER-576
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-576
> Project: Zookeeper
>  Issue Type: Bug
>Reporter: Mahadev konar
>Assignee: Benjamin Reed
> Fix For: 3.2.2, 3.3.0
>
> Attachments: ZOOKEEPER-576.patch
>
>
> the handling and implications of session moved exception should be documented.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-576) docs need to be updated for session moved exception and how to handle it

[Benjamin Reed (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Reed updated ZOOKEEPER-576:


Status: Patch Available  (was: Open)

> docs need to be updated for session moved exception and how to handle it
> 
>
> Key: ZOOKEEPER-576
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-576
> Project: Zookeeper
>  Issue Type: Bug
>Reporter: Mahadev konar
>Assignee: Benjamin Reed
> Fix For: 3.2.2, 3.3.0
>
> Attachments: ZOOKEEPER-576.patch
>
>
> the handling and implications of session moved exception should be documented.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (ZOOKEEPER-582) ZooKeeper can revert to old data when a snapshot is created outside of normal processing

[Mahadev konar (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780327#action_12780327
 ] 

Mahadev konar commented on ZOOKEEPER-582:
-

for 1) good catch.. i missed that

for 2) good pointill fix that  

> ZooKeeper can revert to old data when a snapshot is created outside of normal 
> processing
> 
>
> Key: ZOOKEEPER-582
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-582
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.1.1, 3.2.1
>Reporter: Benjamin Reed
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 3.2.2, 3.3.0, 3.1.2
>
> Attachments: test.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, 
> ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582_3.1.patch, 
> ZOOKEEPER-582_3.2.patch
>
>
> when zookeeper starts up it will restore the most recent state (latest zxid) 
> it finds in the data directory. unfortunately, in the quorum version of 
> zookeeper updates are logged using an epoch based on the latest log file in a 
> directory. if there is a snapshot with a higher epoch than the log files, the 
> zookeeper server will start logging using an epoch one higher than the 
> highest log file.
> so if a data directory has a snapshot with an epoch of 27 and there are no 
> log files, zookeeper will start logging changes using epoch 1. if the cluster 
> restarts the state will be restored from the snapshot with the epoch of 27, 
> which in effect, restores old data.
> normal operation of zookeeper will never result in this situation.
> this does not effect standalone zookeeper.
> a fix should make sure to use an epoch one higher than the current state, 
> whether it comes from the snapshot or log, and should include a sanity check 
> to make sure that a follower never connects to a leader that has a lower 
> epoch than its own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-582) ZooKeeper can revert to old data when a snapshot is created outside of normal processing

[Benjamin Reed (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Reed updated ZOOKEEPER-582:


Status: Open  (was: Patch Available)

looks good mahadev just two things:

1) (minor) in getLastLoggedZxid() you should be useing maxLogZxid instead of 
calling getLastLoggedZxid() again.

2) when doing the sanity check with the leaders zxid you should be checking 
epochs not zxids. it is possible for a follower to see something later and have 
to truncate from the same epoch, put a follower should never see a later epoch.

> ZooKeeper can revert to old data when a snapshot is created outside of normal 
> processing
> 
>
> Key: ZOOKEEPER-582
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-582
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.2.1, 3.1.1
>Reporter: Benjamin Reed
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 3.2.2, 3.3.0, 3.1.2
>
> Attachments: test.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, 
> ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582_3.1.patch, 
> ZOOKEEPER-582_3.2.patch
>
>
> when zookeeper starts up it will restore the most recent state (latest zxid) 
> it finds in the data directory. unfortunately, in the quorum version of 
> zookeeper updates are logged using an epoch based on the latest log file in a 
> directory. if there is a snapshot with a higher epoch than the log files, the 
> zookeeper server will start logging using an epoch one higher than the 
> highest log file.
> so if a data directory has a snapshot with an epoch of 27 and there are no 
> log files, zookeeper will start logging changes using epoch 1. if the cluster 
> restarts the state will be restored from the snapshot with the epoch of 27, 
> which in effect, restores old data.
> normal operation of zookeeper will never result in this situation.
> this does not effect standalone zookeeper.
> a fix should make sure to use an epoch one higher than the current state, 
> whether it comes from the snapshot or log, and should include a sanity check 
> to make sure that a follower never connects to a leader that has a lower 
> epoch than its own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (ZOOKEEPER-582) ZooKeeper can revert to old data when a snapshot is created outside of normal processing

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780323#action_12780323
 ] 

Hadoop QA commented on ZOOKEEPER-582:
-

+1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12425538/ZOOKEEPER-582.patch
  against trunk revision 882313.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 9 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/69/testReport/
Findbugs warnings: 
http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/69/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/69/console

This message is automatically generated.

> ZooKeeper can revert to old data when a snapshot is created outside of normal 
> processing
> 
>
> Key: ZOOKEEPER-582
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-582
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.1.1, 3.2.1
>Reporter: Benjamin Reed
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 3.2.2, 3.3.0, 3.1.2
>
> Attachments: test.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, 
> ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582_3.1.patch, 
> ZOOKEEPER-582_3.2.patch
>
>
> when zookeeper starts up it will restore the most recent state (latest zxid) 
> it finds in the data directory. unfortunately, in the quorum version of 
> zookeeper updates are logged using an epoch based on the latest log file in a 
> directory. if there is a snapshot with a higher epoch than the log files, the 
> zookeeper server will start logging using an epoch one higher than the 
> highest log file.
> so if a data directory has a snapshot with an epoch of 27 and there are no 
> log files, zookeeper will start logging changes using epoch 1. if the cluster 
> restarts the state will be restored from the snapshot with the epoch of 27, 
> which in effect, restores old data.
> normal operation of zookeeper will never result in this situation.
> this does not effect standalone zookeeper.
> a fix should make sure to use an epoch one higher than the current state, 
> whether it comes from the snapshot or log, and should include a sanity check 
> to make sure that a follower never connects to a leader that has a lower 
> epoch than its own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (ZOOKEEPER-582) ZooKeeper can revert to old data when a snapshot is created outside of normal processing

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780317#action_12780317
 ] 

Hadoop QA commented on ZOOKEEPER-582:
-

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12425536/ZOOKEEPER-582.patch
  against trunk revision 882313.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 9 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/68/testReport/
Findbugs warnings: 
http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/68/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/68/console

This message is automatically generated.

> ZooKeeper can revert to old data when a snapshot is created outside of normal 
> processing
> 
>
> Key: ZOOKEEPER-582
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-582
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.1.1, 3.2.1
>Reporter: Benjamin Reed
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 3.2.2, 3.3.0, 3.1.2
>
> Attachments: test.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, 
> ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582_3.1.patch, 
> ZOOKEEPER-582_3.2.patch
>
>
> when zookeeper starts up it will restore the most recent state (latest zxid) 
> it finds in the data directory. unfortunately, in the quorum version of 
> zookeeper updates are logged using an epoch based on the latest log file in a 
> directory. if there is a snapshot with a higher epoch than the log files, the 
> zookeeper server will start logging using an epoch one higher than the 
> highest log file.
> so if a data directory has a snapshot with an epoch of 27 and there are no 
> log files, zookeeper will start logging changes using epoch 1. if the cluster 
> restarts the state will be restored from the snapshot with the epoch of 27, 
> which in effect, restores old data.
> normal operation of zookeeper will never result in this situation.
> this does not effect standalone zookeeper.
> a fix should make sure to use an epoch one higher than the current state, 
> whether it comes from the snapshot or log, and should include a sanity check 
> to make sure that a follower never connects to a leader that has a lower 
> epoch than its own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-582) ZooKeeper can revert to old data when a snapshot is created outside of normal processing

[Mahadev konar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mahadev konar updated ZOOKEEPER-582:


Status: Patch Available  (was: Open)

> ZooKeeper can revert to old data when a snapshot is created outside of normal 
> processing
> 
>
> Key: ZOOKEEPER-582
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-582
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.2.1, 3.1.1
>Reporter: Benjamin Reed
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 3.2.2, 3.3.0, 3.1.2
>
> Attachments: test.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, 
> ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582_3.1.patch, 
> ZOOKEEPER-582_3.2.patch
>
>
> when zookeeper starts up it will restore the most recent state (latest zxid) 
> it finds in the data directory. unfortunately, in the quorum version of 
> zookeeper updates are logged using an epoch based on the latest log file in a 
> directory. if there is a snapshot with a higher epoch than the log files, the 
> zookeeper server will start logging using an epoch one higher than the 
> highest log file.
> so if a data directory has a snapshot with an epoch of 27 and there are no 
> log files, zookeeper will start logging changes using epoch 1. if the cluster 
> restarts the state will be restored from the snapshot with the epoch of 27, 
> which in effect, restores old data.
> normal operation of zookeeper will never result in this situation.
> this does not effect standalone zookeeper.
> a fix should make sure to use an epoch one higher than the current state, 
> whether it comes from the snapshot or log, and should include a sanity check 
> to make sure that a follower never connects to a leader that has a lower 
> epoch than its own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-582) ZooKeeper can revert to old data when a snapshot is created outside of normal processing

[Mahadev konar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mahadev konar updated ZOOKEEPER-582:


Attachment: ZOOKEEPER-582.patch

looks like my eclipse settings added tabs to the indenation. fixed it in this 
patch.

> ZooKeeper can revert to old data when a snapshot is created outside of normal 
> processing
> 
>
> Key: ZOOKEEPER-582
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-582
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.1.1, 3.2.1
>Reporter: Benjamin Reed
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 3.2.2, 3.3.0, 3.1.2
>
> Attachments: test.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, 
> ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582_3.1.patch, 
> ZOOKEEPER-582_3.2.patch
>
>
> when zookeeper starts up it will restore the most recent state (latest zxid) 
> it finds in the data directory. unfortunately, in the quorum version of 
> zookeeper updates are logged using an epoch based on the latest log file in a 
> directory. if there is a snapshot with a higher epoch than the log files, the 
> zookeeper server will start logging using an epoch one higher than the 
> highest log file.
> so if a data directory has a snapshot with an epoch of 27 and there are no 
> log files, zookeeper will start logging changes using epoch 1. if the cluster 
> restarts the state will be restored from the snapshot with the epoch of 27, 
> which in effect, restores old data.
> normal operation of zookeeper will never result in this situation.
> this does not effect standalone zookeeper.
> a fix should make sure to use an epoch one higher than the current state, 
> whether it comes from the snapshot or log, and should include a sanity check 
> to make sure that a follower never connects to a leader that has a lower 
> epoch than its own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-582) ZooKeeper can revert to old data when a snapshot is created outside of normal processing

[Mahadev konar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mahadev konar updated ZOOKEEPER-582:


Status: Open  (was: Patch Available)

> ZooKeeper can revert to old data when a snapshot is created outside of normal 
> processing
> 
>
> Key: ZOOKEEPER-582
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-582
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.2.1, 3.1.1
>Reporter: Benjamin Reed
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 3.2.2, 3.3.0, 3.1.2
>
> Attachments: test.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, 
> ZOOKEEPER-582.patch, ZOOKEEPER-582_3.1.patch, ZOOKEEPER-582_3.2.patch
>
>
> when zookeeper starts up it will restore the most recent state (latest zxid) 
> it finds in the data directory. unfortunately, in the quorum version of 
> zookeeper updates are logged using an epoch based on the latest log file in a 
> directory. if there is a snapshot with a higher epoch than the log files, the 
> zookeeper server will start logging using an epoch one higher than the 
> highest log file.
> so if a data directory has a snapshot with an epoch of 27 and there are no 
> log files, zookeeper will start logging changes using epoch 1. if the cluster 
> restarts the state will be restored from the snapshot with the epoch of 27, 
> which in effect, restores old data.
> normal operation of zookeeper will never result in this situation.
> this does not effect standalone zookeeper.
> a fix should make sure to use an epoch one higher than the current state, 
> whether it comes from the snapshot or log, and should include a sanity check 
> to make sure that a follower never connects to a leader that has a lower 
> epoch than its own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-582) ZooKeeper can revert to old data when a snapshot is created outside of normal processing

[Mahadev konar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mahadev konar updated ZOOKEEPER-582:


Status: Patch Available  (was: Open)

> ZooKeeper can revert to old data when a snapshot is created outside of normal 
> processing
> 
>
> Key: ZOOKEEPER-582
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-582
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.2.1, 3.1.1
>Reporter: Benjamin Reed
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 3.2.2, 3.3.0, 3.1.2
>
> Attachments: test.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, 
> ZOOKEEPER-582.patch, ZOOKEEPER-582_3.1.patch, ZOOKEEPER-582_3.2.patch
>
>
> when zookeeper starts up it will restore the most recent state (latest zxid) 
> it finds in the data directory. unfortunately, in the quorum version of 
> zookeeper updates are logged using an epoch based on the latest log file in a 
> directory. if there is a snapshot with a higher epoch than the log files, the 
> zookeeper server will start logging using an epoch one higher than the 
> highest log file.
> so if a data directory has a snapshot with an epoch of 27 and there are no 
> log files, zookeeper will start logging changes using epoch 1. if the cluster 
> restarts the state will be restored from the snapshot with the epoch of 27, 
> which in effect, restores old data.
> normal operation of zookeeper will never result in this situation.
> this does not effect standalone zookeeper.
> a fix should make sure to use an epoch one higher than the current state, 
> whether it comes from the snapshot or log, and should include a sanity check 
> to make sure that a follower never connects to a leader that has a lower 
> epoch than its own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-582) ZooKeeper can revert to old data when a snapshot is created outside of normal processing

[Mahadev konar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mahadev konar updated ZOOKEEPER-582:


Status: Open  (was: Patch Available)

> ZooKeeper can revert to old data when a snapshot is created outside of normal 
> processing
> 
>
> Key: ZOOKEEPER-582
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-582
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.2.1, 3.1.1
>Reporter: Benjamin Reed
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 3.2.2, 3.3.0, 3.1.2
>
> Attachments: test.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, 
> ZOOKEEPER-582.patch, ZOOKEEPER-582_3.1.patch, ZOOKEEPER-582_3.2.patch
>
>
> when zookeeper starts up it will restore the most recent state (latest zxid) 
> it finds in the data directory. unfortunately, in the quorum version of 
> zookeeper updates are logged using an epoch based on the latest log file in a 
> directory. if there is a snapshot with a higher epoch than the log files, the 
> zookeeper server will start logging using an epoch one higher than the 
> highest log file.
> so if a data directory has a snapshot with an epoch of 27 and there are no 
> log files, zookeeper will start logging changes using epoch 1. if the cluster 
> restarts the state will be restored from the snapshot with the epoch of 27, 
> which in effect, restores old data.
> normal operation of zookeeper will never result in this situation.
> this does not effect standalone zookeeper.
> a fix should make sure to use an epoch one higher than the current state, 
> whether it comes from the snapshot or log, and should include a sanity check 
> to make sure that a follower never connects to a leader that has a lower 
> epoch than its own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-582) ZooKeeper can revert to old data when a snapshot is created outside of normal processing

[Mahadev konar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mahadev konar updated ZOOKEEPER-582:


Attachment: ZOOKEEPER-582.patch

this patch fixes the issue with FLE test. Ill upload the other patches for 3.1 
and 3.2 as soon as hudson is done running this.

> ZooKeeper can revert to old data when a snapshot is created outside of normal 
> processing
> 
>
> Key: ZOOKEEPER-582
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-582
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.1.1, 3.2.1
>Reporter: Benjamin Reed
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 3.2.2, 3.3.0, 3.1.2
>
> Attachments: test.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, 
> ZOOKEEPER-582.patch, ZOOKEEPER-582_3.1.patch, ZOOKEEPER-582_3.2.patch
>
>
> when zookeeper starts up it will restore the most recent state (latest zxid) 
> it finds in the data directory. unfortunately, in the quorum version of 
> zookeeper updates are logged using an epoch based on the latest log file in a 
> directory. if there is a snapshot with a higher epoch than the log files, the 
> zookeeper server will start logging using an epoch one higher than the 
> highest log file.
> so if a data directory has a snapshot with an epoch of 27 and there are no 
> log files, zookeeper will start logging changes using epoch 1. if the cluster 
> restarts the state will be restored from the snapshot with the epoch of 27, 
> which in effect, restores old data.
> normal operation of zookeeper will never result in this situation.
> this does not effect standalone zookeeper.
> a fix should make sure to use an epoch one higher than the current state, 
> whether it comes from the snapshot or log, and should include a sanity check 
> to make sure that a follower never connects to a leader that has a lower 
> epoch than its own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-570) AsyncHammerTest is broken, callbacks need to validate rc parameter

[Patrick Hunt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt updated ZOOKEEPER-570:
---

Fix Version/s: (was: 3.2.2)

Dropping this from 3.2.2, it's not a bug fix (well, it is a fix to a test) but 
it does depend on refactored code from
3.3, it's not clear how this would be done (easily/successfully), dropping from 
3.2.2

> AsyncHammerTest is broken, callbacks need to validate rc parameter
> --
>
> Key: ZOOKEEPER-570
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-570
> Project: Zookeeper
>  Issue Type: Bug
>  Components: tests
>Affects Versions: 3.2.1
>Reporter: Patrick Hunt
>Assignee: Patrick Hunt
>Priority: Critical
> Fix For: 3.3.0
>
> Attachments: ZOOKEEPER-570.patch, ZOOKEEPER-570.patch
>
>
> the asynchammertest is not validating the rc in the callback, more serious is 
> that it is using path in the create callback
> to delete the node, rather than name (which is important in the case of a 
> sequential node creation as in this case)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (ZOOKEEPER-425) Add OSGi metadata to zookeeper.jar

[David Bosschaert (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780278#action_12780278
 ] 

David Bosschaert commented on ZOOKEEPER-425:


Let me explain the context of my work. For the OSGi Remote Services Admin 
Service (an OSGi specification based on the Distributed OSGi RFC 119 work) we 
are working on the Reference Implementation in CXF 
(http://cxf.apache.org/distributed-osgi.html). The Reference Implementation 
supports the Discovery concepts of the Remote Service specification and for the 
implementation we make use of ZooKeeper.

In our code, we use zookeeper in two ways: 
* In zookeeper client code. This code resides in bundles that import the 
org.apache.zookeeper and org.apache.zookeeper.data
* We also need to be able to run the zookeeper server from inside an OSGi bundle

So this is really cases 1 & 3 above.

I think we all agree on 1, and I can certainly put that in a separate patch.

On part 2 - this is the one that Alan filed ZOOKEEPER-584 for, right? This is a 
case that I don't need at this point, and therefore I probably don't understand 
what the best way to realize it is. I understand that this can be quite useful 
and registering a 'trivial' zookeeper object in the service registry is easy, 
but I guess what Alan is really looking for is an object from which he can get 
some management information. I really see this as a separate issue and one 
possibly best solved by someone who actually has a need for this.

The code that I attached does address part 3 somewhat, although I agree that it 
could be better. However it's a start. The zookeeper server starts and can be 
configured. I can try to make the shutdown/restart on reconfiguration work as 
well. I haven't been troubled by the System.exits() yet. It works well enough 
for me at the moment, although it could be improved. Could we not do the 
improvement over time in the future, starting off with something that works a 
bit like in my patch and then making it better over time?

Patrick, on your comment re a contrib package. First of all the code is not 
specific to an osgi container and should work equally in all osgi frameworks. 
Do you mean just putting the source code in a separate location or are you also 
thinking of putting it in a separate jar file? I would suggest putting the 
Activator and ManagedService in the ordinary zookeeper.jar. Outside of an OSGi 
framework these classes are simply ignored, but when the jar is used as an OSGi 
bundle they are automatically found and used. If you put it in a separate jar 
you will have to deploy 2 bundles before you can run the zookeeper server in an 
OSGi framework, which is possible but you'd have to export 
org.apache.zookeeper.server which is currently internal to make it work. 
Besides these two classes are currently only about 5kb in total.

> Add OSGi metadata to zookeeper.jar
> --
>
> Key: ZOOKEEPER-425
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-425
> Project: Zookeeper
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.1.1
>Reporter: David Bosschaert
> Attachments: MANIFEST.MF, zk_patch3.patch
>
>
> After adding OSGi metadata to zookeeper.jar it can be used as both an OSGi 
> bundle as well as an ordinary jar file. 
> In the CXF/DOSGi project the buildsystem does this using the 
> maven-bundle-plugin: 
> http://svn.apache.org/repos/asf/cxf/dosgi/trunk/discovery/distributed/zookeeper-wrapper/pom.xml
> The MANIFEST.MF generated by maven-bundle-plugin is attached to this bug, 
> this works for the CXF/DOSGi project.
> If your buildsystem isn't using maven, I would advise to use bnd 
> (http://www.aqute.biz/Code/Bnd). BND defines its own ant task in which you 
> should be able to use more or less the same instructions as were used in 
> maven:
> 
>   ZooKeeper bundle
>   This bundle contains the ZooKeeper 
> library
>   org.apache.hadoop.zookeeper
>   3.1.1
>   *
>   *;version=3.1.1
> 
> Oh and one other thing. Is it really necessary to put the source code in the 
> Jar file too? I would put that in a separate source distribution :)
> See also: 
> http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-user/200905.mbox/%3c4a2009b1.3030...@yahoo-inc.com%3e

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Fix releases 3.2.2 and 3.1.2 are in progress

[Patrick Hunt]

Due to the issue I reported a few days ago, ZOOKEEPER-582 (email 
attached), we will be creating new 3.2.2. and 3.1.2 releases to address. 
Release candidates should be put up for the vote in the next day or so.


3.2.2
https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&&pid=12310801&fixfor=12314335&sorter/field=issuekey&sorter/order=DESC&sorter/field=priority&sorter/order=DESC

3.1.2
https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&&pid=12310801&fixfor=12314394&sorter/field=issuekey&sorter/order=DESC&sorter/field=priority&sorter/order=DESC

Patrick
--- Begin Message ---
We've found an issue with the migration tool used to migrate users from 
version 2 of ZooKeeper to version 3. This tool was provided to users who 
upgraded from the SourceForge v2 ZK, after we moved to being a 
subproject of Apache Hadoop (which is the same time that we incremented 
the version number from 2.x.y to 3.0.0 - 2+ years ago)


https://issues.apache.org/jira/browse/ZOOKEEPER-582

Please note, this only effects users who migrated their ZK data 
directory from version 2 to version 3. If you have only ever used ZK 
version 3.0.0 and later this does not effect you.


Patrick
--- End Message ---

[jira] Updated: (ZOOKEEPER-562) c client can flood server with pings if tcp send queue filled

[Patrick Hunt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt updated ZOOKEEPER-562:
---

Fix Version/s: 3.1.2

> c client can flood server with pings if tcp send queue filled
> -
>
> Key: ZOOKEEPER-562
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-562
> Project: Zookeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.2.1
>Reporter: Patrick Hunt
>Assignee: Benjamin Reed
>Priority: Blocker
> Fix For: 3.2.2, 3.3.0, 3.1.2
>
> Attachments: ZOOKEEPER-562.patch
>
>
> The c client can flood the server with pings if the tcp queue is filled.
> Say the cluster is overloaded and shuts down the recv processing
> a c client can send a ping, but since last_send is only updated on successful 
> pushing of data into the 
> socket, if flush_send_queue fails to send any data (send_buffer returns 0) 
> then last_send is not updated
> and zookeeper_interest will again send a ping the next time it is woken - 
> which could be 0 if recv_to is close
> to 0, easily could happen if server is not sending data to the client.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (ZOOKEEPER-425) Add OSGi metadata to zookeeper.jar

[Patrick Hunt (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780227#action_12780227
 ] 

Patrick Hunt commented on ZOOKEEPER-425:


1) would go into build.xml

2) & 3) would be in a contrib package, correct? src/contrib/osgi -- or is this 
stuff typically implementation
specific such as src/contrib/felix ? (assuming not the latter)


> Add OSGi metadata to zookeeper.jar
> --
>
> Key: ZOOKEEPER-425
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-425
> Project: Zookeeper
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.1.1
>Reporter: David Bosschaert
> Attachments: MANIFEST.MF, zk_patch3.patch
>
>
> After adding OSGi metadata to zookeeper.jar it can be used as both an OSGi 
> bundle as well as an ordinary jar file. 
> In the CXF/DOSGi project the buildsystem does this using the 
> maven-bundle-plugin: 
> http://svn.apache.org/repos/asf/cxf/dosgi/trunk/discovery/distributed/zookeeper-wrapper/pom.xml
> The MANIFEST.MF generated by maven-bundle-plugin is attached to this bug, 
> this works for the CXF/DOSGi project.
> If your buildsystem isn't using maven, I would advise to use bnd 
> (http://www.aqute.biz/Code/Bnd). BND defines its own ant task in which you 
> should be able to use more or less the same instructions as were used in 
> maven:
> 
>   ZooKeeper bundle
>   This bundle contains the ZooKeeper 
> library
>   org.apache.hadoop.zookeeper
>   3.1.1
>   *
>   *;version=3.1.1
> 
> Oh and one other thing. Is it really necessary to put the source code in the 
> Jar file too? I would put that in a separate source distribution :)
> See also: 
> http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-user/200905.mbox/%3c4a2009b1.3030...@yahoo-inc.com%3e

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-582) ZooKeeper can revert to old data when a snapshot is created outside of normal processing

[Patrick Hunt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt updated ZOOKEEPER-582:
---

Fix Version/s: 3.3.0

> ZooKeeper can revert to old data when a snapshot is created outside of normal 
> processing
> 
>
> Key: ZOOKEEPER-582
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-582
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.1.1, 3.2.1
>Reporter: Benjamin Reed
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 3.2.2, 3.3.0, 3.1.2
>
> Attachments: test.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, 
> ZOOKEEPER-582_3.1.patch, ZOOKEEPER-582_3.2.patch
>
>
> when zookeeper starts up it will restore the most recent state (latest zxid) 
> it finds in the data directory. unfortunately, in the quorum version of 
> zookeeper updates are logged using an epoch based on the latest log file in a 
> directory. if there is a snapshot with a higher epoch than the log files, the 
> zookeeper server will start logging using an epoch one higher than the 
> highest log file.
> so if a data directory has a snapshot with an epoch of 27 and there are no 
> log files, zookeeper will start logging changes using epoch 1. if the cluster 
> restarts the state will be restored from the snapshot with the epoch of 27, 
> which in effect, restores old data.
> normal operation of zookeeper will never result in this situation.
> this does not effect standalone zookeeper.
> a fix should make sure to use an epoch one higher than the current state, 
> whether it comes from the snapshot or log, and should include a sanity check 
> to make sure that a follower never connects to a leader that has a lower 
> epoch than its own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (ZOOKEEPER-425) Add OSGi metadata to zookeeper.jar

[Benjamin Reed (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780219#action_12780219
 ] 

Benjamin Reed commented on ZOOKEEPER-425:
-

david, nice work. i think we should break this patch into possibly three parts:

1) manifest update to just export the right packages
2) an activator that registers a ZooKeeper object as a service. (this would be 
a client object)
3) an activator that starts up a server.

you seem to be ignoring 2). it might be useful to have the bundles of an osgi 
framework share a ZooKeeper object. maybe not though. sharing makes it nice to 
get a preconfigured connected ZooKeeper object from the service registry.

we can't really support 3) properly right now. it is possible to shutdown a 
server and start it back up. the interfaces aren't very well exposed but the 
tests do it. a bigger problem is that we have System.exits in our code, and 
even if the bundle doesn't have permission to call System.exit, not exiting can 
cause bad things to happen.

i would suggest focusing this issue on 1) and possibly 2), but leave 3) for a 
separate issue.

> Add OSGi metadata to zookeeper.jar
> --
>
> Key: ZOOKEEPER-425
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-425
> Project: Zookeeper
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.1.1
>Reporter: David Bosschaert
> Attachments: MANIFEST.MF, zk_patch3.patch
>
>
> After adding OSGi metadata to zookeeper.jar it can be used as both an OSGi 
> bundle as well as an ordinary jar file. 
> In the CXF/DOSGi project the buildsystem does this using the 
> maven-bundle-plugin: 
> http://svn.apache.org/repos/asf/cxf/dosgi/trunk/discovery/distributed/zookeeper-wrapper/pom.xml
> The MANIFEST.MF generated by maven-bundle-plugin is attached to this bug, 
> this works for the CXF/DOSGi project.
> If your buildsystem isn't using maven, I would advise to use bnd 
> (http://www.aqute.biz/Code/Bnd). BND defines its own ant task in which you 
> should be able to use more or less the same instructions as were used in 
> maven:
> 
>   ZooKeeper bundle
>   This bundle contains the ZooKeeper 
> library
>   org.apache.hadoop.zookeeper
>   3.1.1
>   *
>   *;version=3.1.1
> 
> Oh and one other thing. Is it really necessary to put the source code in the 
> Jar file too? I would put that in a separate source distribution :)
> See also: 
> http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-user/200905.mbox/%3c4a2009b1.3030...@yahoo-inc.com%3e

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Assigned: (ZOOKEEPER-582) ZooKeeper can revert to old data when a snapshot is created outside of normal processing

[Patrick Hunt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt reassigned ZOOKEEPER-582:
--

Assignee: Mahadev konar

> ZooKeeper can revert to old data when a snapshot is created outside of normal 
> processing
> 
>
> Key: ZOOKEEPER-582
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-582
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.1.1, 3.2.1
>Reporter: Benjamin Reed
>Assignee: Mahadev konar
>Priority: Blocker
> Fix For: 3.2.2, 3.1.2
>
> Attachments: test.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, 
> ZOOKEEPER-582_3.1.patch, ZOOKEEPER-582_3.2.patch
>
>
> when zookeeper starts up it will restore the most recent state (latest zxid) 
> it finds in the data directory. unfortunately, in the quorum version of 
> zookeeper updates are logged using an epoch based on the latest log file in a 
> directory. if there is a snapshot with a higher epoch than the log files, the 
> zookeeper server will start logging using an epoch one higher than the 
> highest log file.
> so if a data directory has a snapshot with an epoch of 27 and there are no 
> log files, zookeeper will start logging changes using epoch 1. if the cluster 
> restarts the state will be restored from the snapshot with the epoch of 27, 
> which in effect, restores old data.
> normal operation of zookeeper will never result in this situation.
> this does not effect standalone zookeeper.
> a fix should make sure to use an epoch one higher than the current state, 
> whether it comes from the snapshot or log, and should include a sanity check 
> to make sure that a follower never connects to a leader that has a lower 
> epoch than its own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-420) build/test should not require install in zkpython

[Patrick Hunt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt updated ZOOKEEPER-420:
---

Fix Version/s: 3.2.2

> build/test should not require install in zkpython
> -
>
> Key: ZOOKEEPER-420
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-420
> Project: Zookeeper
>  Issue Type: Bug
>  Components: contrib-bindings
>Reporter: Patrick Hunt
>Assignee: Henry Robinson
> Fix For: 3.2.2, 3.3.0
>
> Attachments: build.jpg, ZOOKEEPER-420.patch, ZOOKEEPER-420.patch, 
> ZOOKEEPER-420.patch, ZOOKEEPER-420.patch, ZOOKEEPER-420.patch
>
>
> Currently you cannot just build and test the zkpython contrib, you need to 
> actually install the zookeeper client c library as well
> as the zkpython lib itself.
> There really needs to be 2 steps:
> 1) build/test zkpython "encapsulated" within the src repository, there should 
> be no requirement to actually install anything
> (this is esp the case for automated processes and for review by PMC during 
> release time for example)
> 2) build an egg that can be distributed/installed by end user

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-538) zookeeper.async causes python to segfault

[Patrick Hunt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt updated ZOOKEEPER-538:
---

Fix Version/s: 3.2.2

> zookeeper.async causes python to segfault
> -
>
> Key: ZOOKEEPER-538
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-538
> Project: Zookeeper
>  Issue Type: Bug
>  Components: contrib-bindings
>Affects Versions: 3.2.1
>Reporter: Patrick Hunt
>Assignee: Henry Robinson
>Priority: Critical
> Fix For: 3.2.2, 3.3.0
>
> Attachments: callback.patch, callback.patch
>
>
> Henry, can you take a look at this, am I doing it right?
> calling 
> zookeeper.async(self.handle, path)
> causes python to segfault.
> see: http://github.com/phunt/zk-smoketest/blob/master/zk-smoketest.py

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-510) zkpython lumps all exceptions as IOError, needs specialized exceptions for KeeperException types

[Patrick Hunt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt updated ZOOKEEPER-510:
---

Fix Version/s: 3.2.2

> zkpython lumps all exceptions as IOError, needs specialized exceptions for 
> KeeperException types
> 
>
> Key: ZOOKEEPER-510
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-510
> Project: Zookeeper
>  Issue Type: Bug
>  Components: contrib-bindings
>Affects Versions: 3.2.0
>Reporter: Patrick Hunt
>Assignee: Henry Robinson
> Fix For: 3.2.2, 3.3.0
>
> Attachments: ZOOKEEPER-510-incref.patch, ZOOKEEPER-510.patch, 
> ZOOKEEPER-510.patch, ZOOKEEPER-510.patch, ZOOKEEPER-510.patch
>
>
> The current zkpython bindings always throw "IOError("text")" exceptions, even 
> for ZK specific exceptions such as NODEEXISTS. This makes it difficult (error 
> prone) to handle exceptions in python code. You can't easily pickup a 
> connection loss vs a node exists for example. Of course you could match the 
> error string, but this seems like a bad idea imo.
> We need to add specific exception types to the python binding that map 
> directly to KeeperException/java types. It would also be useful to include 
> the information provided by the KeeperException (like path in some cases), 
> etc... as part of the error thrown to the python code. Would probably be a 
> good idea to stay as close to java api as possible wrt mapping the errors.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-554) zkpython can segfault when statting a deleted node

[Patrick Hunt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt updated ZOOKEEPER-554:
---

Fix Version/s: 3.2.2

> zkpython can segfault when statting a deleted node
> --
>
> Key: ZOOKEEPER-554
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-554
> Project: Zookeeper
>  Issue Type: Bug
>  Components: contrib-bindings
>Reporter: Henry Robinson
>Assignee: Henry Robinson
> Fix For: 3.2.2, 3.3.0
>
> Attachments: ZOOKEEPER-554.patch
>
>
> C client returns NULL for stat object for deleted nodes. zookeeper.c blindly 
> dereferences it. Segfault. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-541) zkpython limited to 256 handles

[Patrick Hunt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt updated ZOOKEEPER-541:
---

Fix Version/s: 3.2.2

> zkpython limited to 256 handles
> ---
>
> Key: ZOOKEEPER-541
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-541
> Project: Zookeeper
>  Issue Type: Bug
>  Components: contrib-bindings
>Affects Versions: 3.2.1
>Reporter: Patrick Hunt
>Assignee: Henry Robinson
> Fix For: 3.2.2, 3.3.0
>
> Attachments: ZOOKEEPER-541.patch
>
>
> zkpython is currently limited to a max of 256 total handles - not 256 open 
> handles, but rather 256 total handles created
> over the lifetime of the python application.
> In general this isn't a real issue, however in the case of a long lived 
> application which polls the cluster periodically (closing
> the session btw calls) this is an issue.
> it would be great if the slots could be reused? or perhaps a more complex 
> structure, such as a linked list, which would allow
> dynamic growth/shrinkage of the handle list.
> Also see ZOOKEEPER-540

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-540) zkpython needs better tracking of handle validity

[Patrick Hunt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt updated ZOOKEEPER-540:
---

Fix Version/s: 3.2.2

> zkpython needs better tracking of handle validity
> -
>
> Key: ZOOKEEPER-540
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-540
> Project: Zookeeper
>  Issue Type: Bug
>  Components: contrib-bindings
>Affects Versions: 3.2.1
>Reporter: Patrick Hunt
>Assignee: Henry Robinson
> Fix For: 3.2.2, 3.3.0
>
>
> I was getting a python segfault in one of my scripts. Turns out I was closing 
> a session handle and then reusing it (async call). This was causing python to 
> segfault.
> zkpython should track handle state and complain, rather than crash, if the 
> handle is invalid (closed).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (ZOOKEEPER-582) ZooKeeper can revert to old data when a snapshot is created outside of normal processing

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780177#action_12780177
 ] 

Hadoop QA commented on ZOOKEEPER-582:
-

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12425503/ZOOKEEPER-582.patch
  against trunk revision 881882.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 9 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/67/testReport/
Findbugs warnings: 
http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/67/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/67/console

This message is automatically generated.

> ZooKeeper can revert to old data when a snapshot is created outside of normal 
> processing
> 
>
> Key: ZOOKEEPER-582
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-582
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.1.1, 3.2.1
>Reporter: Benjamin Reed
>Priority: Blocker
> Fix For: 3.2.2, 3.1.2
>
> Attachments: test.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, 
> ZOOKEEPER-582_3.1.patch, ZOOKEEPER-582_3.2.patch
>
>
> when zookeeper starts up it will restore the most recent state (latest zxid) 
> it finds in the data directory. unfortunately, in the quorum version of 
> zookeeper updates are logged using an epoch based on the latest log file in a 
> directory. if there is a snapshot with a higher epoch than the log files, the 
> zookeeper server will start logging using an epoch one higher than the 
> highest log file.
> so if a data directory has a snapshot with an epoch of 27 and there are no 
> log files, zookeeper will start logging changes using epoch 1. if the cluster 
> restarts the state will be restored from the snapshot with the epoch of 27, 
> which in effect, restores old data.
> normal operation of zookeeper will never result in this situation.
> this does not effect standalone zookeeper.
> a fix should make sure to use an epoch one higher than the current state, 
> whether it comes from the snapshot or log, and should include a sanity check 
> to make sure that a follower never connects to a leader that has a lower 
> epoch than its own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Re: Build failed in Hudson: ZooKeeper-trunk #544

[Patrick Hunt]

It looks to me to be the same issue we've been seeing lately on hudson 
trunk - testHammer fails randomly. We've tried a bunch of things to 
reproduce but so far no luck. Even running the test on the hudson 
machine with the same settings (outside hudson) we can't reproduce. We 
haven't seen this failure on the patch queue, which is even weirder.


afaict the session creation is stalling in the request processor, but 
it's not clear where. We may need to create a jira with some 
instrumented test code to see if we can get more data when it fails on 
hudson. Take a look though, maybe you'll see something I did not.


Patrick

Henry Robinson wrote:

The failed test is testObserversHammer - I'm having a look to see what might
be wrong.

Henry

On Thu, Nov 19, 2009 at 2:57 AM, Apache Hudson Server <
hud...@hudson.zones.apache.org> wrote:


See  inline with the variable.

[breed] ZOOKEEPER-519. Followerhandler should close the socket if it gets
an exception on a write.

[breed] ZOOKEEPER-532. java compiler should be target Java 1.5

--
[...truncated 98279 lines...]
   [junit] 2009-11-19 10:57:31,578 - INFO
 [main-SendThread(localhost:11225):clientcnxn$sendthr...@929] - Opening
socket connection to server localhost/127.0.0.1:11225
   [junit] 2009-11-19 10:57:31,578 - INFO
 [main-SendThread(localhost:11225):clientcnxn$sendthr...@837] - Socket
connection established to localhost/127.0.0.1:11225, initiating session
   [junit] 2009-11-19 10:57:31,579 - INFO
 [NIOServerCxn.Factory:11225:nioservercnxn$fact...@214] - Accepted socket
connection from /127.0.0.1:54182
   [junit] 2009-11-19 10:57:31,579 - INFO
 [NIOServerCxn.Factory:11225:nioserverc...@688] - Client attempting to
renew session 0x1250c16f121 at /127.0.0.1:54182
   [junit] 2009-11-19 10:57:31,580 - INFO
 [NIOServerCxn.Factory:11225:nioserverc...@1131] - Established session
0x1250c16f121 for client /127.0.0.1:54182
   [junit] 2009-11-19 10:57:31,580 - INFO
 [main-SendThread(localhost:11225):clientcnxn$sendthr...@640] - Session
establishment complete, sessionid = 0x1250c16f121
   [junit] 2009-11-19 10:57:31,588 - INFO  [main:clientb...@376] -
STOPPING server
   [junit] 2009-11-19 10:57:31,589 - INFO  [main:nioserverc...@974] -
Closed socket connection for client /127.0.0.1:54182 which had sessionid
0x1250c16f121
   [junit] 2009-11-19 10:57:31,589 - INFO
 [main-SendThread(localhost:11225):clientcnxn$sendthr...@1047] - Unable to
read additional data from server sessionid 0x1250c16f121, likely server
has closed socket, closing socket connection and attempting reconnect
   [junit] 2009-11-19 10:57:31,590 - INFO
 [NIOServerCxn.Factory:11225:nioservercnxn$fact...@241] - NIOServerCnxn
factory exited run method
   [junit] 2009-11-19 10:57:31,590 - INFO  [main:finalrequestproces...@365]
- shutdown of request processor complete
   [junit] 2009-11-19 10:57:31,590 - INFO
 [SyncThread:0:syncrequestproces...@151] - SyncRequestProcessor exited!
   [junit] 2009-11-19 10:57:31,590 - INFO
 [ProcessThread:-1:preprequestproces...@119] - PrepRequestProcessor exited
loop!
   [junit] ensureOnly:[]
   [junit] 2009-11-19 10:57:31,690 - INFO  [main:clientb...@369] -
STARTING server
   [junit] 2009-11-19 10:57:31,690 - INFO  [main:zookeeperser...@160] -
Created server
   [junit] 2009-11-19 10:57:31,691 - INFO  [main:nioservercnxn$fact...@123]
- binding to port 11225
   [junit] 2009-11-19 10:57:31,692 - INFO  [main:files...@81] - Reading
snapshot <
http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/build/test/tmp/test4406575549066528204.junit.dir/version-2/snapshot.5
   [junit] 2009-11-19 10:57:31,694 - INFO  [main:filetxnsnap...@208] -
Snapshotting: 6
   [junit] 2009-11-19 10:57:31,696 - INFO
 [NIOServerCxn.Factory:11225:nioservercnxn$fact...@214] - Accepted socket
connection from /127.0.0.1:54184
   [junit] 2009-11-19 10:57:31,696 - INFO
 [NIOServerCxn.Factory:11225:nioserverc...@782] - Processing stat command
from /127.0.0.1:54184
   [junit] 2009-11-19 10:57:31,697 - INFO
 [NIOServerCxn.Factory:11225:nioserverc...@974] - Closed socket connection
for client /127.0.0.1:54184 (no session established for client)
   [junit] ensureOnly:[InMemoryDataTree, StandaloneServer_port]
   [junit] expect:InMemoryDataTree
   [junit] found:InMemoryDataTree
org.apache.ZooKeeperService:name0=StandaloneServer_port-1,name1=InMemoryDataTree
   [junit] expect:StandaloneServer_port
   [junit] found:StandaloneServer_port
org.apache.ZooKeeperService:name0=StandaloneServer_port-1
   [junit] 2009-11-19 10:57:33,000 - INFO
 [SessionTracker:sessiontrackeri...@145] - SessionTrackerImpl exited loop!
   [junit] 2009-11-19 10:57:33,586 - INFO
 [main-SendThread(localhost:11225):clientcnxn$sendth

[jira] Updated: (ZOOKEEPER-582) ZooKeeper can revert to old data when a snapshot is created outside of normal processing

[Mahadev konar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mahadev konar updated ZOOKEEPER-582:


Status: Patch Available  (was: Open)

> ZooKeeper can revert to old data when a snapshot is created outside of normal 
> processing
> 
>
> Key: ZOOKEEPER-582
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-582
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.2.1, 3.1.1
>Reporter: Benjamin Reed
>Priority: Blocker
> Fix For: 3.2.2, 3.1.2
>
> Attachments: test.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, 
> ZOOKEEPER-582_3.1.patch, ZOOKEEPER-582_3.2.patch
>
>
> when zookeeper starts up it will restore the most recent state (latest zxid) 
> it finds in the data directory. unfortunately, in the quorum version of 
> zookeeper updates are logged using an epoch based on the latest log file in a 
> directory. if there is a snapshot with a higher epoch than the log files, the 
> zookeeper server will start logging using an epoch one higher than the 
> highest log file.
> so if a data directory has a snapshot with an epoch of 27 and there are no 
> log files, zookeeper will start logging changes using epoch 1. if the cluster 
> restarts the state will be restored from the snapshot with the epoch of 27, 
> which in effect, restores old data.
> normal operation of zookeeper will never result in this situation.
> this does not effect standalone zookeeper.
> a fix should make sure to use an epoch one higher than the current state, 
> whether it comes from the snapshot or log, and should include a sanity check 
> to make sure that a follower never connects to a leader that has a lower 
> epoch than its own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-582) ZooKeeper can revert to old data when a snapshot is created outside of normal processing

[Mahadev konar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mahadev konar updated ZOOKEEPER-582:


Attachment: ZOOKEEPER-582.patch

this patch includes the patch and the test for trunk. ill upload combined 
patches for 3.1 and 3.2 branch.

> ZooKeeper can revert to old data when a snapshot is created outside of normal 
> processing
> 
>
> Key: ZOOKEEPER-582
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-582
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.1.1, 3.2.1
>Reporter: Benjamin Reed
>Priority: Blocker
> Fix For: 3.2.2, 3.1.2
>
> Attachments: test.patch, ZOOKEEPER-582.patch, ZOOKEEPER-582.patch, 
> ZOOKEEPER-582_3.1.patch, ZOOKEEPER-582_3.2.patch
>
>
> when zookeeper starts up it will restore the most recent state (latest zxid) 
> it finds in the data directory. unfortunately, in the quorum version of 
> zookeeper updates are logged using an epoch based on the latest log file in a 
> directory. if there is a snapshot with a higher epoch than the log files, the 
> zookeeper server will start logging using an epoch one higher than the 
> highest log file.
> so if a data directory has a snapshot with an epoch of 27 and there are no 
> log files, zookeeper will start logging changes using epoch 1. if the cluster 
> restarts the state will be restored from the snapshot with the epoch of 27, 
> which in effect, restores old data.
> normal operation of zookeeper will never result in this situation.
> this does not effect standalone zookeeper.
> a fix should make sure to use an epoch one higher than the current state, 
> whether it comes from the snapshot or log, and should include a sanity check 
> to make sure that a follower never connects to a leader that has a lower 
> epoch than its own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Re: Build failed in Hudson: ZooKeeper-trunk #544

[Henry Robinson]

The failed test is testObserversHammer - I'm having a look to see what might
be wrong.

Henry

On Thu, Nov 19, 2009 at 2:57 AM, Apache Hudson Server <
hud...@hudson.zones.apache.org> wrote:

> See  >
>
> Changes:
>
> [mahadev] ZOOKEEPER-368. Observers: core functionality (henry robinson via
> mahadev)
>
> [breed] ZOOKEEPER-3. syncLimit has slightly different comments in the class
> header, and > inline with the variable.
>
> [breed] ZOOKEEPER-519. Followerhandler should close the socket if it gets
> an exception on a write.
>
> [breed] ZOOKEEPER-532. java compiler should be target Java 1.5
>
> --
> [...truncated 98279 lines...]
>[junit] 2009-11-19 10:57:31,578 - INFO
>  [main-SendThread(localhost:11225):clientcnxn$sendthr...@929] - Opening
> socket connection to server localhost/127.0.0.1:11225
>[junit] 2009-11-19 10:57:31,578 - INFO
>  [main-SendThread(localhost:11225):clientcnxn$sendthr...@837] - Socket
> connection established to localhost/127.0.0.1:11225, initiating session
>[junit] 2009-11-19 10:57:31,579 - INFO
>  [NIOServerCxn.Factory:11225:nioservercnxn$fact...@214] - Accepted socket
> connection from /127.0.0.1:54182
>[junit] 2009-11-19 10:57:31,579 - INFO
>  [NIOServerCxn.Factory:11225:nioserverc...@688] - Client attempting to
> renew session 0x1250c16f121 at /127.0.0.1:54182
>[junit] 2009-11-19 10:57:31,580 - INFO
>  [NIOServerCxn.Factory:11225:nioserverc...@1131] - Established session
> 0x1250c16f121 for client /127.0.0.1:54182
>[junit] 2009-11-19 10:57:31,580 - INFO
>  [main-SendThread(localhost:11225):clientcnxn$sendthr...@640] - Session
> establishment complete, sessionid = 0x1250c16f121
>[junit] 2009-11-19 10:57:31,588 - INFO  [main:clientb...@376] -
> STOPPING server
>[junit] 2009-11-19 10:57:31,589 - INFO  [main:nioserverc...@974] -
> Closed socket connection for client /127.0.0.1:54182 which had sessionid
> 0x1250c16f121
>[junit] 2009-11-19 10:57:31,589 - INFO
>  [main-SendThread(localhost:11225):clientcnxn$sendthr...@1047] - Unable to
> read additional data from server sessionid 0x1250c16f121, likely server
> has closed socket, closing socket connection and attempting reconnect
>[junit] 2009-11-19 10:57:31,590 - INFO
>  [NIOServerCxn.Factory:11225:nioservercnxn$fact...@241] - NIOServerCnxn
> factory exited run method
>[junit] 2009-11-19 10:57:31,590 - INFO  [main:finalrequestproces...@365]
> - shutdown of request processor complete
>[junit] 2009-11-19 10:57:31,590 - INFO
>  [SyncThread:0:syncrequestproces...@151] - SyncRequestProcessor exited!
>[junit] 2009-11-19 10:57:31,590 - INFO
>  [ProcessThread:-1:preprequestproces...@119] - PrepRequestProcessor exited
> loop!
>[junit] ensureOnly:[]
>[junit] 2009-11-19 10:57:31,690 - INFO  [main:clientb...@369] -
> STARTING server
>[junit] 2009-11-19 10:57:31,690 - INFO  [main:zookeeperser...@160] -
> Created server
>[junit] 2009-11-19 10:57:31,691 - INFO  [main:nioservercnxn$fact...@123]
> - binding to port 11225
>[junit] 2009-11-19 10:57:31,692 - INFO  [main:files...@81] - Reading
> snapshot <
> http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/ws/trunk/build/test/tmp/test4406575549066528204.junit.dir/version-2/snapshot.5
> >
>[junit] 2009-11-19 10:57:31,694 - INFO  [main:filetxnsnap...@208] -
> Snapshotting: 6
>[junit] 2009-11-19 10:57:31,696 - INFO
>  [NIOServerCxn.Factory:11225:nioservercnxn$fact...@214] - Accepted socket
> connection from /127.0.0.1:54184
>[junit] 2009-11-19 10:57:31,696 - INFO
>  [NIOServerCxn.Factory:11225:nioserverc...@782] - Processing stat command
> from /127.0.0.1:54184
>[junit] 2009-11-19 10:57:31,697 - INFO
>  [NIOServerCxn.Factory:11225:nioserverc...@974] - Closed socket connection
> for client /127.0.0.1:54184 (no session established for client)
>[junit] ensureOnly:[InMemoryDataTree, StandaloneServer_port]
>[junit] expect:InMemoryDataTree
>[junit] found:InMemoryDataTree
> org.apache.ZooKeeperService:name0=StandaloneServer_port-1,name1=InMemoryDataTree
>[junit] expect:StandaloneServer_port
>[junit] found:StandaloneServer_port
> org.apache.ZooKeeperService:name0=StandaloneServer_port-1
>[junit] 2009-11-19 10:57:33,000 - INFO
>  [SessionTracker:sessiontrackeri...@145] - SessionTrackerImpl exited loop!
>[junit] 2009-11-19 10:57:33,586 - INFO
>  [main-SendThread(localhost:11225):clientcnxn$sendthr...@929] - Opening
> socket connection to server localhost/127.0.0.1:11225
>[junit] 2009-11-19 10:57:33,586 - INFO
>  [NIOServerCxn.Factory:11225:nioservercnxn$fact...@214] - Accepted socket
> connection from /127.0.0.1:54185
>[junit] 2009-11-19 10:57:33,586 - INFO
>  [main-SendThread(localhost:11225):clientcnxn$sendthr...@837] - Socket
> connection established to localhost/127.0.0.1:11225, initiating session
>[junit] 2009-11-19 10:57:33,587 - INFO
>  [NIOServerCxn

[jira] Commented: (ZOOKEEPER-425) Add OSGi metadata to zookeeper.jar

[Alan Cabrera (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780024#action_12780024
 ] 

Alan Cabrera commented on ZOOKEEPER-425:


Sure.  I saw that we were adding a bundle activator and deployment via 
Configuration Admin Service so I wasn't sure where we were drawing the line 
here.  Issue added.

> Add OSGi metadata to zookeeper.jar
> --
>
> Key: ZOOKEEPER-425
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-425
> Project: Zookeeper
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.1.1
>Reporter: David Bosschaert
> Attachments: MANIFEST.MF, zk_patch3.patch
>
>
> After adding OSGi metadata to zookeeper.jar it can be used as both an OSGi 
> bundle as well as an ordinary jar file. 
> In the CXF/DOSGi project the buildsystem does this using the 
> maven-bundle-plugin: 
> http://svn.apache.org/repos/asf/cxf/dosgi/trunk/discovery/distributed/zookeeper-wrapper/pom.xml
> The MANIFEST.MF generated by maven-bundle-plugin is attached to this bug, 
> this works for the CXF/DOSGi project.
> If your buildsystem isn't using maven, I would advise to use bnd 
> (http://www.aqute.biz/Code/Bnd). BND defines its own ant task in which you 
> should be able to use more or less the same instructions as were used in 
> maven:
> 
>   ZooKeeper bundle
>   This bundle contains the ZooKeeper 
> library
>   org.apache.hadoop.zookeeper
>   3.1.1
>   *
>   *;version=3.1.1
> 
> Oh and one other thing. Is it really necessary to put the source code in the 
> Jar file too? I would put that in a separate source distribution :)
> See also: 
> http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-user/200905.mbox/%3c4a2009b1.3030...@yahoo-inc.com%3e

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (ZOOKEEPER-584) ZooKeeper service instance should be registered in the OSGi registry

[Alan Cabrera (JIRA)]

ZooKeeper service instance should be registered in the OSGi registry


 Key: ZOOKEEPER-584
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-584
 Project: Zookeeper
  Issue Type: New Feature
  Components: server
Reporter: Alan Cabrera


When Zookeeper is booted in an OSGi framework by {{ManagedService}} it would be 
quite handy to have {{ManagedService}} register a management interface for that 
ZooKeeper instance into the OSGi service registry.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (ZOOKEEPER-425) Add OSGi metadata to zookeeper.jar

[David Bosschaert (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780012#action_12780012
 ] 

David Bosschaert commented on ZOOKEEPER-425:


Hi Alan, that sounds like a useful idea to me and using a service in the OSGi 
Service Registry sounds like a really good approach but I personally think that 
this should be a separate bug/enhancement. Let's get the basics working first :)

> Add OSGi metadata to zookeeper.jar
> --
>
> Key: ZOOKEEPER-425
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-425
> Project: Zookeeper
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.1.1
>Reporter: David Bosschaert
> Attachments: MANIFEST.MF, zk_patch3.patch
>
>
> After adding OSGi metadata to zookeeper.jar it can be used as both an OSGi 
> bundle as well as an ordinary jar file. 
> In the CXF/DOSGi project the buildsystem does this using the 
> maven-bundle-plugin: 
> http://svn.apache.org/repos/asf/cxf/dosgi/trunk/discovery/distributed/zookeeper-wrapper/pom.xml
> The MANIFEST.MF generated by maven-bundle-plugin is attached to this bug, 
> this works for the CXF/DOSGi project.
> If your buildsystem isn't using maven, I would advise to use bnd 
> (http://www.aqute.biz/Code/Bnd). BND defines its own ant task in which you 
> should be able to use more or less the same instructions as were used in 
> maven:
> 
>   ZooKeeper bundle
>   This bundle contains the ZooKeeper 
> library
>   org.apache.hadoop.zookeeper
>   3.1.1
>   *
>   *;version=3.1.1
> 
> Oh and one other thing. Is it really necessary to put the source code in the 
> Jar file too? I would put that in a separate source distribution :)
> See also: 
> http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-user/200905.mbox/%3c4a2009b1.3030...@yahoo-inc.com%3e

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (ZOOKEEPER-425) Add OSGi metadata to zookeeper.jar

[Alan Cabrera (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780001#action_12780001
 ] 

Alan Cabrera commented on ZOOKEEPER-425:


I would like to be informed when the Zookeeper service is started/stopped in 
the OSGi framework so that I can perform some "extender" like tasks in tandem.  
Would it be a good idea to publish a management interface to the service 
registry when Zookeeper is started by the {{ManagedService}}?

> Add OSGi metadata to zookeeper.jar
> --
>
> Key: ZOOKEEPER-425
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-425
> Project: Zookeeper
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.1.1
>Reporter: David Bosschaert
> Attachments: MANIFEST.MF, zk_patch3.patch
>
>
> After adding OSGi metadata to zookeeper.jar it can be used as both an OSGi 
> bundle as well as an ordinary jar file. 
> In the CXF/DOSGi project the buildsystem does this using the 
> maven-bundle-plugin: 
> http://svn.apache.org/repos/asf/cxf/dosgi/trunk/discovery/distributed/zookeeper-wrapper/pom.xml
> The MANIFEST.MF generated by maven-bundle-plugin is attached to this bug, 
> this works for the CXF/DOSGi project.
> If your buildsystem isn't using maven, I would advise to use bnd 
> (http://www.aqute.biz/Code/Bnd). BND defines its own ant task in which you 
> should be able to use more or less the same instructions as were used in 
> maven:
> 
>   ZooKeeper bundle
>   This bundle contains the ZooKeeper 
> library
>   org.apache.hadoop.zookeeper
>   3.1.1
>   *
>   *;version=3.1.1
> 
> Oh and one other thing. Is it really necessary to put the source code in the 
> Jar file too? I would put that in a separate source distribution :)
> See also: 
> http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-user/200905.mbox/%3c4a2009b1.3030...@yahoo-inc.com%3e

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (ZOOKEEPER-425) Add OSGi metadata to zookeeper.jar

[David Bosschaert (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Bosschaert updated ZOOKEEPER-425:
---

Attachment: zk_patch3.patch

The attached patch (named zk_patch3.patch) implements some of the 
functionality. Its not fully finished yet, I'm attaching it at this point to 
get some feedback.

The patch:
* Adds the OSGi Metadata to the MANIFEST.MF
* Can be used as just a library bundle if you want to write a zookeeper client 
in another OSGi bundle.
* Can be used to run the zookeeper server. To do this the server has to be 
configured. Configuration is handled through the OSGi Configuration Admin 
service. The bundle has an optional dependency on this service and registers an 
org.osgi.service.cm.ManagedService service to be configured by CM. The PID for 
this service is org.apache.zookeeper and all of the configuration variables are 
the same as in the non-OSGi zoo.cfg file (it uses the same ServerConfig class 
to handle them). 
All configuration properties have defaults provided, except for the 
'clientPort' one, which has to be provided through Config Admin. For the data 
directory the OSGi Bundle-private storage directory is used by default.
* The jar has an OSGi BundleActivator class, which will be triggered if run on 
an OSGi framework.
* BTW the zookeeper.jar stil also works as a POJ (plain old Jar ;)

note that I updated the ivy.xml file to pull in two OSGi jars that are needed 
at build time. However they don't need to be redistributed with the zookeeper 
jar as they are provided by the OSGi framework.

There are still a few open questions that I have:
* In OSGi things should really be (re)configurable at runtime. This means that 
the Configuration Admin Service may call your ManagedService.updated() callback 
with changed properties at any time. I guess the easiest (but not necessarily 
the most elegant) way to handle this is by taking down the ZooKeeperServerMain 
and relaunching it with the modified properties. ZooKeeperServerMain has a 
shutdown() method that I could use for this. Would that be the best idea?
* I'm not (yet) familiar with the cluster setup and am wondering whether the 
configuration approach that I took also works in that case.
I also intend to provide some unit tests before closing this off.

BTW I haven't gone into detail how the OSGi Configuration Admin service 
typically works. Let me know if I need to provide more info on that.

> Add OSGi metadata to zookeeper.jar
> --
>
> Key: ZOOKEEPER-425
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-425
> Project: Zookeeper
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.1.1
>Reporter: David Bosschaert
> Attachments: MANIFEST.MF, zk_patch3.patch
>
>
> After adding OSGi metadata to zookeeper.jar it can be used as both an OSGi 
> bundle as well as an ordinary jar file. 
> In the CXF/DOSGi project the buildsystem does this using the 
> maven-bundle-plugin: 
> http://svn.apache.org/repos/asf/cxf/dosgi/trunk/discovery/distributed/zookeeper-wrapper/pom.xml
> The MANIFEST.MF generated by maven-bundle-plugin is attached to this bug, 
> this works for the CXF/DOSGi project.
> If your buildsystem isn't using maven, I would advise to use bnd 
> (http://www.aqute.biz/Code/Bnd). BND defines its own ant task in which you 
> should be able to use more or less the same instructions as were used in 
> maven:
> 
>   ZooKeeper bundle
>   This bundle contains the ZooKeeper 
> library
>   org.apache.hadoop.zookeeper
>   3.1.1
>   *
>   *;version=3.1.1
> 
> Oh and one other thing. Is it really necessary to put the source code in the 
> Jar file too? I would put that in a separate source distribution :)
> See also: 
> http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-user/200905.mbox/%3c4a2009b1.3030...@yahoo-inc.com%3e

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Build failed in Hudson: ZooKeeper-trunk #544

[Apache Hudson Server]

See 

Changes:

[mahadev] ZOOKEEPER-368. Observers: core functionality (henry robinson via 
mahadev)

[breed] ZOOKEEPER-3. syncLimit has slightly different comments in the class 
header, and > inline with the variable.

[breed] ZOOKEEPER-519. Followerhandler should close the socket if it gets an 
exception on a write.

[breed] ZOOKEEPER-532. java compiler should be target Java 1.5

--
[...truncated 98279 lines...]
[junit] 2009-11-19 10:57:31,578 - INFO  
[main-SendThread(localhost:11225):clientcnxn$sendthr...@929] - Opening socket 
connection to server localhost/127.0.0.1:11225
[junit] 2009-11-19 10:57:31,578 - INFO  
[main-SendThread(localhost:11225):clientcnxn$sendthr...@837] - Socket 
connection established to localhost/127.0.0.1:11225, initiating session
[junit] 2009-11-19 10:57:31,579 - INFO  
[NIOServerCxn.Factory:11225:nioservercnxn$fact...@214] - Accepted socket 
connection from /127.0.0.1:54182
[junit] 2009-11-19 10:57:31,579 - INFO  
[NIOServerCxn.Factory:11225:nioserverc...@688] - Client attempting to renew 
session 0x1250c16f121 at /127.0.0.1:54182
[junit] 2009-11-19 10:57:31,580 - INFO  
[NIOServerCxn.Factory:11225:nioserverc...@1131] - Established session 
0x1250c16f121 for client /127.0.0.1:54182
[junit] 2009-11-19 10:57:31,580 - INFO  
[main-SendThread(localhost:11225):clientcnxn$sendthr...@640] - Session 
establishment complete, sessionid = 0x1250c16f121
[junit] 2009-11-19 10:57:31,588 - INFO  [main:clientb...@376] - STOPPING 
server
[junit] 2009-11-19 10:57:31,589 - INFO  [main:nioserverc...@974] - Closed 
socket connection for client /127.0.0.1:54182 which had sessionid 
0x1250c16f121
[junit] 2009-11-19 10:57:31,589 - INFO  
[main-SendThread(localhost:11225):clientcnxn$sendthr...@1047] - Unable to read 
additional data from server sessionid 0x1250c16f121, likely server has 
closed socket, closing socket connection and attempting reconnect
[junit] 2009-11-19 10:57:31,590 - INFO  
[NIOServerCxn.Factory:11225:nioservercnxn$fact...@241] - NIOServerCnxn factory 
exited run method
[junit] 2009-11-19 10:57:31,590 - INFO  [main:finalrequestproces...@365] - 
shutdown of request processor complete
[junit] 2009-11-19 10:57:31,590 - INFO  
[SyncThread:0:syncrequestproces...@151] - SyncRequestProcessor exited!
[junit] 2009-11-19 10:57:31,590 - INFO  
[ProcessThread:-1:preprequestproces...@119] - PrepRequestProcessor exited loop!
[junit] ensureOnly:[]
[junit] 2009-11-19 10:57:31,690 - INFO  [main:clientb...@369] - STARTING 
server
[junit] 2009-11-19 10:57:31,690 - INFO  [main:zookeeperser...@160] - 
Created server
[junit] 2009-11-19 10:57:31,691 - INFO  [main:nioservercnxn$fact...@123] - 
binding to port 11225
[junit] 2009-11-19 10:57:31,692 - INFO  [main:files...@81] - Reading 
snapshot 

[junit] 2009-11-19 10:57:31,694 - INFO  [main:filetxnsnap...@208] - 
Snapshotting: 6
[junit] 2009-11-19 10:57:31,696 - INFO  
[NIOServerCxn.Factory:11225:nioservercnxn$fact...@214] - Accepted socket 
connection from /127.0.0.1:54184
[junit] 2009-11-19 10:57:31,696 - INFO  
[NIOServerCxn.Factory:11225:nioserverc...@782] - Processing stat command from 
/127.0.0.1:54184
[junit] 2009-11-19 10:57:31,697 - INFO  
[NIOServerCxn.Factory:11225:nioserverc...@974] - Closed socket connection for 
client /127.0.0.1:54184 (no session established for client)
[junit] ensureOnly:[InMemoryDataTree, StandaloneServer_port]
[junit] expect:InMemoryDataTree
[junit] found:InMemoryDataTree 
org.apache.ZooKeeperService:name0=StandaloneServer_port-1,name1=InMemoryDataTree
[junit] expect:StandaloneServer_port
[junit] found:StandaloneServer_port 
org.apache.ZooKeeperService:name0=StandaloneServer_port-1
[junit] 2009-11-19 10:57:33,000 - INFO  
[SessionTracker:sessiontrackeri...@145] - SessionTrackerImpl exited loop!
[junit] 2009-11-19 10:57:33,586 - INFO  
[main-SendThread(localhost:11225):clientcnxn$sendthr...@929] - Opening socket 
connection to server localhost/127.0.0.1:11225
[junit] 2009-11-19 10:57:33,586 - INFO  
[NIOServerCxn.Factory:11225:nioservercnxn$fact...@214] - Accepted socket 
connection from /127.0.0.1:54185
[junit] 2009-11-19 10:57:33,586 - INFO  
[main-SendThread(localhost:11225):clientcnxn$sendthr...@837] - Socket 
connection established to localhost/127.0.0.1:11225, initiating session
[junit] 2009-11-19 10:57:33,587 - INFO  
[NIOServerCxn.Factory:11225:nioserverc...@688] - Client attempting to renew 
session 0x1250c16f121 at /127.0.0.1:54185
[junit] 2009-11-19 10:57:33,587 - INFO  
[NIOServerCxn.Factory:11225:nioserverc...@1131] - Established session 
0x1250c16f121 for client /127.0.0.1:54185
[junit] 2009-11-19 10:57:3