[jira] [Commented] (ZOOKEEPER-938) Support Kerberos authentication of clients.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086086#comment-13086086 ] Hadoop QA commented on ZOOKEEPER-938: - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12490160/ZOOKEEPER-938.patch against trunk revision 1157698. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/463//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/463//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/463//console This message is automatically generated. > Support Kerberos authentication of clients. > --- > > Key: ZOOKEEPER-938 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-938 > Project: ZooKeeper > Issue Type: New Feature > Components: java client, server >Reporter: Eugene Koontz >Assignee: Eugene Koontz > Fix For: 3.4.0, 3.5.0 > > Attachments: NIOServerCnxn.patch, ZOOKEEPER-938.patch, > ZOOKEEPER-938.patch, ZOOKEEPER-938.patch, ZOOKEEPER-938.patch, > ZOOKEEPER-938.patch, ZOOKEEPER-938.patch, ZOOKEEPER-938.patch, > ZOOKEEPER-938.patch, ZOOKEEPER-938.patch, ZOOKEEPER-938.patch, > ZOOKEEPER-938.patch, ZOOKEEPER-938.patch, ZOOKEEPER-938.patch, > ZOOKEEPER-938.patch, jaas.conf, sasl.patch > > > Support Kerberos authentication of clients. > The following usage would let an admin use Kerberos authentication to assign > ACLs to authenticated clients. > 1. Admin logs into zookeeper (not necessarily through Kerberos however). > 2. Admin decides that a new node called '/mynode' should be owned by the user > 'zkclient' and have full permissions on this. > 3. Admin does: zk> create /mynode content sasl:zkcli...@foofers.org:cdrwa > 4. User 'zkclient' logins to kerberos using the command line utility 'kinit'. > 5. User connects to zookeeper server using a Kerberos-enabled version of > zkClient (ZookeeperMain). > 6. Behind the scenes, the client and server exchange authentication > information. User is now authenticated as 'zkclient'. > 7. User accesses /mynode with permissions 'cdrwa'. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1154) Data inconsistency when the node(s) with the highest zxid is not present at the time of leader election
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086060#comment-13086060 ] Mahadev konar commented on ZOOKEEPER-1154: -- Good catch Vishal. I agree with Ben. n = 1 should be good enough. It'd be the most common case. > Data inconsistency when the node(s) with the highest zxid is not present at > the time of leader election > --- > > Key: ZOOKEEPER-1154 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1154 > Project: ZooKeeper > Issue Type: Bug > Components: quorum >Affects Versions: 3.3.3 >Reporter: Vishal Kathuria >Priority: Blocker > Fix For: 3.4.0 > > Original Estimate: 504h > Remaining Estimate: 504h > > If a participant with the highest zxid (lets call it A) isn't present during > leader election, a participant with a lower zxid (say B) might be chosen as a > leader. When A comes up, it will replay the log with that higher zxid. The > change that was in that higher zxid will only be visible to the clients > connecting to the participant A, but not to other participants. > I was able to reproduce this problem by > 1. connect debugger to B and C and suspend them, so they don't write anything > 2. Issue an update to the leader A. > 3. After a few seconds, crash all servers (A,B,C) > 4. Start B and C, let the leader election take place > 5. Start A. > 6. You will find that the update done in step 2 is visible on A but not on > B,C, hence the inconsistency. > Below is a more detailed analysis of what is happening in the code. > Initial Condition > 1.Lets say there are three nodes in the ensemble A,B,C with A being the > leader > 2.The current epoch is 7. > 3.For simplicity of the example, lets say zxid is a two digit number, > with epoch being the first digit. > 4.The zxid is 73 > 5.All the nodes have seen the change 73 and have persistently logged it. > Step 1 > Request with zxid 74 is issued. The leader A writes it to the log but there > is a crash of the entire ensemble and B,C never write the change 74 to their > log. > Step 3 > B,C restart, A is still down > B,C form the quorum > B is the new leader. Lets say B minCommitLog is 71 and maxCommitLog is 73 > epoch is now 8, zxid is 80 > Request with zxid 81 is successful. On B, minCommitLog is now 71, > maxCommitLog is 81 > Step 4 > A starts up. It applies the change in request with zxid 74 to its in-memory > data tree > A contacts B to registerAsFollower and provides 74 as its ZxId > Since 71<=74<=81, B decides to send A the diff. B will send to A the proposal > 81. > Problem: > The problem with the above sequence is that A's data tree has the update from > request 74, which is not correct. Before getting the proposals 81, A should > have received a trunc to 73. I don't see that in the code. If the > maxCommitLog on B hadn't bumped to 81 but had stayed at 73, that case seems > to be fine. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1154) Data inconsistency when the node(s) with the highest zxid is not present at the time of leader election
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085886#comment-13085886 ] Vishal Kathuria commented on ZOOKEEPER-1154: Thanks Ben, Yes, n=1 should be easy to implement but I was worried it would lead to more frequent sending of snaps instead of diffs. This could be a problem for ZK installations with large data (some of which are several GB). Looking at the code to see the best way to implement this for the general case - I should have an update soon. thanks! > Data inconsistency when the node(s) with the highest zxid is not present at > the time of leader election > --- > > Key: ZOOKEEPER-1154 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1154 > Project: ZooKeeper > Issue Type: Bug > Components: quorum >Affects Versions: 3.3.3 >Reporter: Vishal Kathuria >Priority: Blocker > Fix For: 3.4.0 > > Original Estimate: 504h > Remaining Estimate: 504h > > If a participant with the highest zxid (lets call it A) isn't present during > leader election, a participant with a lower zxid (say B) might be chosen as a > leader. When A comes up, it will replay the log with that higher zxid. The > change that was in that higher zxid will only be visible to the clients > connecting to the participant A, but not to other participants. > I was able to reproduce this problem by > 1. connect debugger to B and C and suspend them, so they don't write anything > 2. Issue an update to the leader A. > 3. After a few seconds, crash all servers (A,B,C) > 4. Start B and C, let the leader election take place > 5. Start A. > 6. You will find that the update done in step 2 is visible on A but not on > B,C, hence the inconsistency. > Below is a more detailed analysis of what is happening in the code. > Initial Condition > 1.Lets say there are three nodes in the ensemble A,B,C with A being the > leader > 2.The current epoch is 7. > 3.For simplicity of the example, lets say zxid is a two digit number, > with epoch being the first digit. > 4.The zxid is 73 > 5.All the nodes have seen the change 73 and have persistently logged it. > Step 1 > Request with zxid 74 is issued. The leader A writes it to the log but there > is a crash of the entire ensemble and B,C never write the change 74 to their > log. > Step 3 > B,C restart, A is still down > B,C form the quorum > B is the new leader. Lets say B minCommitLog is 71 and maxCommitLog is 73 > epoch is now 8, zxid is 80 > Request with zxid 81 is successful. On B, minCommitLog is now 71, > maxCommitLog is 81 > Step 4 > A starts up. It applies the change in request with zxid 74 to its in-memory > data tree > A contacts B to registerAsFollower and provides 74 as its ZxId > Since 71<=74<=81, B decides to send A the diff. B will send to A the proposal > 81. > Problem: > The problem with the above sequence is that A's data tree has the update from > request 74, which is not correct. Before getting the proposals 81, A should > have received a trunc to 73. I don't see that in the code. If the > maxCommitLog on B hadn't bumped to 81 but had stayed at 73, that case seems > to be fine. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (ZOOKEEPER-1155) Add windows automated builds (CI) for zookeeper c client bindings
Add windows automated builds (CI) for zookeeper c client bindings - Key: ZOOKEEPER-1155 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1155 Project: ZooKeeper Issue Type: Improvement Components: c client Affects Versions: 3.3.4, 3.4.0 Reporter: Dheeraj Agrawal setup an CI build on windows to make sure that the new code checked in compiles fine on windows (VS compilers) for the zookeeper c bindings. There is a ticket opened with the INFRA team to assign a build box and setup CI env for zookeeper c bindings https://issues.apache.org/jira/browse/INFRA-3842 feel free to help us with this effort, this will ensure that the new checkins dont break windows builds. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1108) Various bugs in zoo_add_auth in C
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085730#comment-13085730
]
Camille Fournier commented on ZOOKEEPER-1108:
-
Mahadev, what do you say? We would like to get this into 3.4 since it causes us
a lot of grief with the auth in the C client. I think Dheeraj's point is good,
and the Java and C clients are already divergent enough in this manner that
this shouldn't be an issue.
> Various bugs in zoo_add_auth in C
> -
>
> Key: ZOOKEEPER-1108
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1108
> Project: ZooKeeper
> Issue Type: Bug
> Components: c client
>Affects Versions: 3.3.3
>Reporter: Dheeraj Agrawal
>Assignee: Dheeraj Agrawal
>Priority: Blocker
> Fix For: 3.4.0
>
> Attachments: ZOOKEEPER-1108.patch, ZOOKEEPER-1108.patch,
> ZOOKEEPER-1108.patch, ZOOKEEPER-1108.patch
>
>
> 3 issues:
> In zoo_add_auth: there is a race condition:
>2940 // [ZOOKEEPER-800] zoo_add_auth should return ZINVALIDSTATE if
>2941 // the connection is closed.
>2942 if (zoo_state(zh) == 0) {
>2943 return ZINVALIDSTATE;
>2944 }
> when we do zookeeper_init, the state is initialized to 0 and above we check
> if state = 0 then throw exception.
> There is a race condition where the doIo thread is slow and has not changed
> the state to CONNECTING, then you end up returning back ZKINVALIDSTATE.
> The problem is we use 0 for CLOSED state and UNINITIALIZED state. in case of
> uninitialized case it should let it go through.
> 2nd issue:
> Another Bug: in send_auth_info, the check is not correct
> while (auth->next != NULL) { //--BUG: in cases where there is only one auth
> in the list, this will never send that auth, as its next will be NULL
>rc = send_info_packet(zh, auth);
>auth = auth->next;
> }
> FIX IS:
> do {
> rc = send_info_packet(zh, auth);
> auth = auth->next;
> } while (auth != NULL); //this will make sure that even if there is one auth
> ,that will get sent.
> 3rd issue:
>2965 add_last_auth(&zh->auth_h, authinfo);
>2966 zoo_unlock_auth(zh);
>2967
>2968 if(zh->state == ZOO_CONNECTED_STATE || zh->state ==
> ZOO_ASSOCIATING_STATE)
>2969 return send_last_auth_info(zh);
> if it is connected, we only send the last_auth_info, which may be different
> than the one we added, as we unlocked it before sending it.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1153) Deprecate AuthFLE and LE
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085672#comment-13085672 ] Hadoop QA commented on ZOOKEEPER-1153: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12490526/ZOOKEEPER-1153.patch against trunk revision 1157698. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/462//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/462//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/462//console This message is automatically generated. > Deprecate AuthFLE and LE > > > Key: ZOOKEEPER-1153 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1153 > Project: ZooKeeper > Issue Type: Improvement >Affects Versions: 3.3.3 >Reporter: Flavio Junqueira >Assignee: Flavio Junqueira > Fix For: 3.4.0 > > Attachments: ZOOKEEPER-1153.patch, ZOOKEEPER-1153.patch > > > I propose we mark these as deprecated in 3.4.0 and remove them in the > following release. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1153) Deprecate AuthFLE and LE
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Junqueira updated ZOOKEEPER-1153: Attachment: ZOOKEEPER-1153.patch Some small improvements. > Deprecate AuthFLE and LE > > > Key: ZOOKEEPER-1153 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1153 > Project: ZooKeeper > Issue Type: Improvement >Affects Versions: 3.3.3 >Reporter: Flavio Junqueira >Assignee: Flavio Junqueira > Fix For: 3.4.0 > > Attachments: ZOOKEEPER-1153.patch, ZOOKEEPER-1153.patch > > > I propose we mark these as deprecated in 3.4.0 and remove them in the > following release. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1153) Deprecate AuthFLE and LE
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Junqueira updated ZOOKEEPER-1153: Attachment: ZOOKEEPER-1153.patch It marks three classes as deprecated and modifies documentation to reflect it. > Deprecate AuthFLE and LE > > > Key: ZOOKEEPER-1153 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1153 > Project: ZooKeeper > Issue Type: Improvement >Affects Versions: 3.3.3 >Reporter: Flavio Junqueira >Assignee: Flavio Junqueira > Fix For: 3.4.0 > > Attachments: ZOOKEEPER-1153.patch > > > I propose we mark these as deprecated in 3.4.0 and remove them in the > following release. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1117) zookeeper 3.3.3 fails to build with gcc >= 4.6.1 on Debian/Ubuntu
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085608#comment-13085608 ] Hadoop QA commented on ZOOKEEPER-1117: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12490512/ZOOKEEPER-1117-3.3.3.patch against trunk revision 1157698. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/461//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/461//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/461//console This message is automatically generated. > zookeeper 3.3.3 fails to build with gcc >= 4.6.1 on Debian/Ubuntu > - > > Key: ZOOKEEPER-1117 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1117 > Project: ZooKeeper > Issue Type: Bug > Components: c client >Affects Versions: 3.3.3, 3.4.0 > Environment: Ubuntu Developement Release (11.10/Oneiric Ocelot), > Debian Unstable (sid) >Reporter: James Page >Assignee: James Page >Priority: Minor > Fix For: 3.3.4, 3.4.0 > > Attachments: ZOOKEEPER-1117-3.3.3.patch, ZOOKEEPER-1117.patch, > disable-cc-errors, gcc-4.6-ftbfs.patch, gcc-4.6-ftbfs.patch, > gcc-4.6-ftbfs.patch, gcc-4.6-trunk-fix.patch > > > zookeeper 3.3.3 (and 3.3.1) fails to build on Debian and Ubuntu systems with > gcc >= 4.6.1: > /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT > zookeeper.lo -MD -MP -MF .deps/zookeeper.Tpo -c -o zookeeper.lo `test -f > 'src/zookeeper.c' || echo './'`src/zookeeper.c > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I./include -I./tests > -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT zookeeper.lo -MD -MP -MF > .deps/zookeeper.Tpo -c src/zookeeper.c -fPIC -DPIC -o .libs/zookeeper.o > src/zookeeper.c: In function 'getaddrs': > src/zookeeper.c:455:13: error: variable 'port' set but not used > [-Werror=unused-but-set-variable] > cc1: all warnings being treated as errors > See http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=625441 for more > information. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1117) zookeeper 3.3.3 fails to build with gcc >= 4.6.1 on Debian/Ubuntu
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Page updated ZOOKEEPER-1117: -- Attachment: ZOOKEEPER-1117-3.3.3.patch ZOOKEEPER-1117.patch Patches for trunk and 3.3.x branch > zookeeper 3.3.3 fails to build with gcc >= 4.6.1 on Debian/Ubuntu > - > > Key: ZOOKEEPER-1117 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1117 > Project: ZooKeeper > Issue Type: Bug > Components: c client >Affects Versions: 3.3.3, 3.4.0 > Environment: Ubuntu Developement Release (11.10/Oneiric Ocelot), > Debian Unstable (sid) >Reporter: James Page >Assignee: James Page >Priority: Minor > Fix For: 3.3.4, 3.4.0 > > Attachments: ZOOKEEPER-1117-3.3.3.patch, ZOOKEEPER-1117.patch, > disable-cc-errors, gcc-4.6-ftbfs.patch, gcc-4.6-ftbfs.patch, > gcc-4.6-ftbfs.patch, gcc-4.6-trunk-fix.patch > > > zookeeper 3.3.3 (and 3.3.1) fails to build on Debian and Ubuntu systems with > gcc >= 4.6.1: > /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I./include -I./tests -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT > zookeeper.lo -MD -MP -MF .deps/zookeeper.Tpo -c -o zookeeper.lo `test -f > 'src/zookeeper.c' || echo './'`src/zookeeper.c > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I./include -I./tests > -I./generated -Wall -Werror -g -O2 -D_GNU_SOURCE -MT zookeeper.lo -MD -MP -MF > .deps/zookeeper.Tpo -c src/zookeeper.c -fPIC -DPIC -o .libs/zookeeper.o > src/zookeeper.c: In function 'getaddrs': > src/zookeeper.c:455:13: error: variable 'port' set but not used > [-Werror=unused-but-set-variable] > cc1: all warnings being treated as errors > See http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=625441 for more > information. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
