[jira] [Commented] (ZOOKEEPER-1748) TCP keepalive for leader election connections

2016-09-28 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531799#comment-15531799
 ] 

ASF GitHub Bot commented on ZOOKEEPER-1748:
---

GitHub user gnethercutt opened a pull request:

https://github.com/apache/zookeeper/pull/83

enable TCP keepalive for the leadership election/quorum socket

Use TCP keep-alives for election/quorum peer connections.

This is the shortest edit distance to address 
[ZOOKEEPER-1748](https://issues.apache.org/jira/browse/ZOOKEEPER-1748), and is 
required to avoid silent packet delivery failures for a long-lived connection 
in AWS (amongst other environments). 

See also:
- [VPC security group connection 
tracking](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html#security-group-connection-tracking)
- [Using TCP 
keepalives](http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/usingkeepalive.html)
- [Zookeeper 
internals](https://zookeeper.apache.org/doc/r3.4.8/zookeeperInternals.html)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gnethercutt/zookeeper election_tcp_keepalive

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/83.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #83


commit bcab41003d91dc121e368337317710c2434bece8
Author: Glenn Nethercutt 
Date:   2016-09-28T18:22:41Z

enable TCP keepalive for the leadership election/quorum socket




> TCP keepalive for leader election connections
> -
>
> Key: ZOOKEEPER-1748
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1748
> Project: ZooKeeper
>  Issue Type: Improvement
>  Components: leaderElection
>Affects Versions: 3.4.5, 3.5.0
> Environment: Linux, Java 1.7
>Reporter: Antal Sasvári
>Assignee: Daniel Peon
>Priority: Minor
> Fix For: 3.5.3, 3.6.0
>
> Attachments: Zookeeper-1748-add_tcp_keepalive.patch
>
>
> In our system we encountered the following problem:
> If the system is stable, and there is no leader election, the leader election 
> port connections are open for very long time without any packets being sent 
> on them.
> Some network elements silently drop the established TCP connection after a 
> timeout if there are no packets being sent on it. In this case the ZK servers 
> will not notice the connection loss. This causes additional delay later when 
> the next leader election is started, as the TCP connections are not alive any 
> more.
> We would like to be able to enable TCP keepalive on the leader election 
> sockets in order to prevent the connection timeout in some network elements 
> due to connection inactivity.
> This could be controlled by adding a new config parameter called tcpKeepAlive 
> in the ZooKeeper configuration file. It would be only applicable in case of 
> algorithm 3 (TCP based fast leader election), having the default value false.
> If tcpKeepAlive is set to true, the TCP keepalive flag should be enabled for 
> the leader election sockets in QuorumCnxManager.setSockOpts() by calling 
> sock.setKeepAlive(true).
> We have tested this change successfully in our environment.
> Please comment whether you see any problem with this. If not, I am going to 
> submit a patch.
> I've been told that e.g. Apache ActiveMQ also has a config option for similar 
> purpose called transport.keepalive.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] zookeeper pull request #83: enable TCP keepalive for the leadership election...

2016-09-28 Thread gnethercutt 
GitHub user gnethercutt opened a pull request:

https://github.com/apache/zookeeper/pull/83

enable TCP keepalive for the leadership election/quorum socket

Use TCP keep-alives for election/quorum peer connections.

This is the shortest edit distance to address 
[ZOOKEEPER-1748](https://issues.apache.org/jira/browse/ZOOKEEPER-1748), and is 
required to avoid silent packet delivery failures for a long-lived connection 
in AWS (amongst other environments). 

See also:
- [VPC security group connection 
tracking](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html#security-group-connection-tracking)
- [Using TCP 
keepalives](http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/usingkeepalive.html)
- [Zookeeper 
internals](https://zookeeper.apache.org/doc/r3.4.8/zookeeperInternals.html)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gnethercutt/zookeeper election_tcp_keepalive

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/83.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #83


commit bcab41003d91dc121e368337317710c2434bece8
Author: Glenn Nethercutt 
Date:   2016-09-28T18:22:41Z

enable TCP keepalive for the leadership election/quorum socket




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (ZOOKEEPER-2014) Only admin should be allowed to reconfig a cluster

2016-09-28 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531585#comment-15531585
 ] 

Hadoop QA commented on ZOOKEEPER-2014:
--

+1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12830832/ZOOKEEPER-2014.patch
  against trunk revision ec20c5434cc8a334b3fd25e27d26dccf4793c8f3.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 40 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 2.0.3) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3459//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3459//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3459//console

This message is automatically generated.

> Only admin should be allowed to reconfig a cluster
> --
>
> Key: ZOOKEEPER-2014
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2014
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.5.0
>Reporter: Raul Gutierrez Segales
>Assignee: Michael Han
>Priority: Blocker
> Fix For: 3.5.3
>
> Attachments: ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, 
> ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, 
> ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch
>
>
> ZOOKEEPER-107 introduces reconfiguration support via the reconfig() call. We 
> should, at the very least, ensure that only the Admin can reconfigure a 
> cluster. Perhaps restricting access to /zookeeper/config as well, though this 
> is debatable. Surely one could ensure Admin only access via an ACL, but that 
> would leave everyone who doesn't use ACLs unprotected. We could also force a 
> default ACL to make it a bit more consistent (maybe).
> Finally, making reconfig() only available to Admins means they have to run 
> with zookeeper.DigestAuthenticationProvider.superDigest (which I am not sure 
> if everyone does, or how would it work with other authentication providers). 
> Review board https://reviews.apache.org/r/51546/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Success: ZOOKEEPER-2014 PreCommit Build #3459

2016-09-28 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/ZOOKEEPER-2014
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3459/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 473642 lines...]
 [exec] 
 [exec] +1 @author.  The patch does not contain any @author tags.
 [exec] 
 [exec] +1 tests included.  The patch appears to include 40 new or 
modified tests.
 [exec] 
 [exec] +1 javadoc.  The javadoc tool did not generate any warning 
messages.
 [exec] 
 [exec] +1 javac.  The applied patch does not increase the total number 
of javac compiler warnings.
 [exec] 
 [exec] +1 findbugs.  The patch does not introduce any new Findbugs 
(version 2.0.3) warnings.
 [exec] 
 [exec] +1 release audit.  The applied patch does not increase the 
total number of release audit warnings.
 [exec] 
 [exec] +1 core tests.  The patch passed core unit tests.
 [exec] 
 [exec] +1 contrib tests.  The patch passed contrib unit tests.
 [exec] 
 [exec] Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3459//testReport/
 [exec] Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3459//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
 [exec] Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3459//console
 [exec] 
 [exec] This message is automatically generated.
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Adding comment to Jira.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] Comment added.
 [exec] 3da1d608b7e106c3ce9223a77bc11f2b30241ea5 logged out
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Finished build.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] mv: 
‘/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-Build/patchprocess’ 
and 
‘/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-Build/patchprocess’ 
are the same file

BUILD SUCCESSFUL
Total time: 18 minutes 35 seconds
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
[description-setter] Description set: ZOOKEEPER-2014
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Success
Sending email for trigger: Success
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###
## FAILED TESTS (if any) 
##
All tests passed

Re: Review Request 51546: ZOOKEEPER-2014: Only admin should be allowed to reconfig a cluster

2016-09-28 Thread Michael Han 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51546/
---

(Updated Sept. 29, 2016, 1:41 a.m.)


Review request for zookeeper, fpj, Patrick Hunt, Raul Gutierrez Segales, and 
Alexander Shraer.


Changes
---

A small fix to previous diff - the new C reconfig test 
testReconfigFailureWithoutAuth was added but not enabled.


Bugs: ZOOKEEPER-2014
https://issues.apache.org/jira/browse/ZOOKEEPER-2014


Repository: zookeeper-git


Description
---

Address various security concerns around reconfig feature (ZOOKEEPER-2014) to 
unblock 3.5.3 release.


Diffs (updated)
-

  build.xml 96c9ef8 
  src/c/include/zookeeper.h 18a203d 
  src/c/tests/TestReconfigServer.cc 6a429ac 
  src/c/tests/ZooKeeperQuorumServer.h aa8b7cc 
  src/c/tests/ZooKeeperQuorumServer.cc 23392cd 
  src/docs/src/documentation/content/xdocs/zookeeperAdmin.xml f729095 
  src/docs/src/documentation/content/xdocs/zookeeperReconfig.xml 7168a01 
  src/java/main/org/apache/zookeeper/ClientCnxn.java 12dd51c 
  src/java/main/org/apache/zookeeper/KeeperException.java a05f1ab 
  src/java/main/org/apache/zookeeper/ZooKeeper.java f2ec3d7 
  src/java/main/org/apache/zookeeper/ZooKeeperMain.java 25d61a4 
  src/java/main/org/apache/zookeeper/admin/ZooKeeperAdmin.java PRE-CREATION 
  src/java/main/org/apache/zookeeper/cli/CliCommand.java 3d0a90b 
  src/java/main/org/apache/zookeeper/cli/ReconfigCommand.java deb7914 
  src/java/main/org/apache/zookeeper/server/DataTree.java 06c80d3 
  src/java/main/org/apache/zookeeper/server/PrepRequestProcessor.java 33f638d 
  src/java/main/org/apache/zookeeper/server/quorum/QuorumPeerConfig.java 
e772fa8 
  src/java/test/org/apache/zookeeper/server/DataTreeTest.java d726643 
  src/java/test/org/apache/zookeeper/server/quorum/ReconfigBackupTest.java 
241af52 
  
src/java/test/org/apache/zookeeper/server/quorum/ReconfigDuringLeaderSyncTest.java
 301837d 
  
src/java/test/org/apache/zookeeper/server/quorum/ReconfigFailureCasesTest.java 
e7147b3 
  src/java/test/org/apache/zookeeper/server/quorum/ReconfigLegacyTest.java 
ee9f2e2 
  src/java/test/org/apache/zookeeper/server/quorum/StandaloneDisabledTest.java 
1f6ce1f 
  src/java/test/org/apache/zookeeper/test/ACLTest.java 9920fc4 
  src/java/test/org/apache/zookeeper/test/ReconfigExceptionTest.java 
PRE-CREATION 
  src/java/test/org/apache/zookeeper/test/ReconfigTest.java 248a754 
  src/java/test/org/apache/zookeeper/test/StandaloneTest.java 5c95280 

Diff: https://reviews.apache.org/r/51546/diff/


Testing
---

Added new Java unit tests that cover various exception cases of using reconfig 
API without proper set up. 
All existing tests (Java and C) passed under stress tests (minors those known 
flaky tests.).
Manuelly tested reconfig commands using ZooKeeper command line tool.


Thanks,

Michael Han

[jira] [Updated] (ZOOKEEPER-2014) Only admin should be allowed to reconfig a cluster

2016-09-28 Thread Michael Han (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Han updated ZOOKEEPER-2014:
---
Attachment: ZOOKEEPER-2014.patch

> Only admin should be allowed to reconfig a cluster
> --
>
> Key: ZOOKEEPER-2014
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2014
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.5.0
>Reporter: Raul Gutierrez Segales
>Assignee: Michael Han
>Priority: Blocker
> Fix For: 3.5.3
>
> Attachments: ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, 
> ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, 
> ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch
>
>
> ZOOKEEPER-107 introduces reconfiguration support via the reconfig() call. We 
> should, at the very least, ensure that only the Admin can reconfigure a 
> cluster. Perhaps restricting access to /zookeeper/config as well, though this 
> is debatable. Surely one could ensure Admin only access via an ACL, but that 
> would leave everyone who doesn't use ACLs unprotected. We could also force a 
> default ACL to make it a bit more consistent (maybe).
> Finally, making reconfig() only available to Admins means they have to run 
> with zookeeper.DigestAuthenticationProvider.superDigest (which I am not sure 
> if everyone does, or how would it work with other authentication providers). 
> Review board https://reviews.apache.org/r/51546/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (ZOOKEEPER-2014) Only admin should be allowed to reconfig a cluster

2016-09-28 Thread Michael Han (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Han updated ZOOKEEPER-2014:
---
Attachment: (was: ZOOKEEPER-2014.patch)

> Only admin should be allowed to reconfig a cluster
> --
>
> Key: ZOOKEEPER-2014
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2014
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.5.0
>Reporter: Raul Gutierrez Segales
>Assignee: Michael Han
>Priority: Blocker
> Fix For: 3.5.3
>
> Attachments: ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, 
> ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, 
> ZOOKEEPER-2014.patch
>
>
> ZOOKEEPER-107 introduces reconfiguration support via the reconfig() call. We 
> should, at the very least, ensure that only the Admin can reconfigure a 
> cluster. Perhaps restricting access to /zookeeper/config as well, though this 
> is debatable. Surely one could ensure Admin only access via an ACL, but that 
> would leave everyone who doesn't use ACLs unprotected. We could also force a 
> default ACL to make it a bit more consistent (maybe).
> Finally, making reconfig() only available to Admins means they have to run 
> with zookeeper.DigestAuthenticationProvider.superDigest (which I am not sure 
> if everyone does, or how would it work with other authentication providers). 
> Review board https://reviews.apache.org/r/51546/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Failed: ZOOKEEPER-2014 PreCommit Build #3458

2016-09-28 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/ZOOKEEPER-2014
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3458/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 469779 lines...]
 [exec] +1 tests included.  The patch appears to include 40 new or 
modified tests.
 [exec] 
 [exec] +1 javadoc.  The javadoc tool did not generate any warning 
messages.
 [exec] 
 [exec] +1 javac.  The applied patch does not increase the total number 
of javac compiler warnings.
 [exec] 
 [exec] +1 findbugs.  The patch does not introduce any new Findbugs 
(version 2.0.3) warnings.
 [exec] 
 [exec] +1 release audit.  The applied patch does not increase the 
total number of release audit warnings.
 [exec] 
 [exec] -1 core tests.  The patch failed core unit tests.
 [exec] 
 [exec] +1 contrib tests.  The patch passed contrib unit tests.
 [exec] 
 [exec] Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3458//testReport/
 [exec] Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3458//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
 [exec] Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3458//console
 [exec] 
 [exec] This message is automatically generated.
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Adding comment to Jira.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] Comment added.
 [exec] d3a5099a447f423d94476f51fd289b5acea19294 logged out
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Finished build.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] mv: 
‘/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-Build/patchprocess’ 
and 
‘/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-Build/patchprocess’ 
are the same file

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-Build/build.xml:1605: 
exec returned: 1

Total time: 18 minutes 41 seconds
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
[description-setter] Description set: ZOOKEEPER-2014
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###
## FAILED TESTS (if any) 
##
All tests passed

[jira] [Commented] (ZOOKEEPER-2014) Only admin should be allowed to reconfig a cluster

2016-09-28 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531402#comment-15531402
 ] 

Hadoop QA commented on ZOOKEEPER-2014:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12830808/ZOOKEEPER-2014.patch
  against trunk revision ec20c5434cc8a334b3fd25e27d26dccf4793c8f3.

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 40 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 2.0.3) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3458//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3458//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3458//console

This message is automatically generated.

> Only admin should be allowed to reconfig a cluster
> --
>
> Key: ZOOKEEPER-2014
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2014
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.5.0
>Reporter: Raul Gutierrez Segales
>Assignee: Michael Han
>Priority: Blocker
> Fix For: 3.5.3
>
> Attachments: ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, 
> ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, 
> ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch
>
>
> ZOOKEEPER-107 introduces reconfiguration support via the reconfig() call. We 
> should, at the very least, ensure that only the Admin can reconfigure a 
> cluster. Perhaps restricting access to /zookeeper/config as well, though this 
> is debatable. Surely one could ensure Admin only access via an ACL, but that 
> would leave everyone who doesn't use ACLs unprotected. We could also force a 
> default ACL to make it a bit more consistent (maybe).
> Finally, making reconfig() only available to Admins means they have to run 
> with zookeeper.DigestAuthenticationProvider.superDigest (which I am not sure 
> if everyone does, or how would it work with other authentication providers). 
> Review board https://reviews.apache.org/r/51546/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-1936) Server exits when unable to create data directory due to race

2016-09-28 Thread Matt Foley (JIRA) 

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531396#comment-15531396
 ] 

Matt Foley commented on ZOOKEEPER-1936:
---

[~cnauroth], please do commit.  +1.  Thanks.

> Server exits when unable to create data directory due to race 
> --
>
> Key: ZOOKEEPER-1936
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1936
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.4.6, 3.5.0
>Reporter: Harald Musum
>Assignee: Andrew Purtell
>Priority: Minor
> Attachments: ZOOKEEPER-1936.patch, ZOOKEEPER-1936.v2.patch, 
> ZOOKEEPER-1936.v3.patch, ZOOKEEPER-1936.v3.patch, ZOOKEEPER-1936.v4.patch
>
>
> We sometime see issues with ZooKeeper server not starting and seeing this 
> error in the log:
> [2014-05-27 09:29:48.248] ERROR   : -   
> .org.apache.zookeeper.server.ZooKeeperServerMainUnexpected exception,
> exiting abnormally\nexception=\njava.io.IOException: Unable to create data
> directory /home/y/var/zookeeper/version-2\n\tat
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.(FileTxnSnapLog.java:85)\n\tat
> org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:103)\n\tat
> org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:86)\n\tat
> org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:52)\n\tat
> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116)\n\tat
> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)\n\t
> [...]
> Stack trace from JVM gives this:
> "PurgeTask" daemon prio=10 tid=0x0201d000 nid=0x1727 runnable
> [0x7f55d7dc7000]
>java.lang.Thread.State: RUNNABLE
> at java.io.UnixFileSystem.createDirectory(Native Method)
> at java.io.File.mkdir(File.java:1310)
> at java.io.File.mkdirs(File.java:1337)
> at
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.(FileTxnSnapLog.java:84)
> at org.apache.zookeeper.server.PurgeTxnLog.purge(PurgeTxnLog.java:68)
> at
> org.apache.zookeeper.server.DatadirCleanupManager$PurgeTask.run(DatadirCleanupManager.java:140)
> at java.util.TimerThread.mainLoop(Timer.java:555)
> at java.util.TimerThread.run(Timer.java:505)
> "zookeeper server" prio=10 tid=0x027df800 nid=0x1715 runnable
> [0x7f55d7ed8000]
>java.lang.Thread.State: RUNNABLE
> at java.io.UnixFileSystem.createDirectory(Native Method)
> at java.io.File.mkdir(File.java:1310)
> at java.io.File.mkdirs(File.java:1337)
> at
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.(FileTxnSnapLog.java:84)
> at
> org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:103)
> at
> org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:86)
> at
> org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:52)
> at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116)
> at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
> [...]
> So it seems that when autopurge is used (as it is in our case), it might 
> happen at the same time as starting the server itself. In FileTxnSnapLog() it 
> will check if the directory exists and create it if not. These two tasks do 
> this at the same time, and mkdir fails and server exits the JVM.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-2014) Only admin should be allowed to reconfig a cluster

2016-09-28 Thread Michael Han (JIRA) 

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531257#comment-15531257
 ] 

Michael Han commented on ZOOKEEPER-2014:


[~fpj]: Sounds good - actually just updated patch that incorporates some C side 
changes. PTAL when you have a chance, thanks a lot!

> Only admin should be allowed to reconfig a cluster
> --
>
> Key: ZOOKEEPER-2014
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2014
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.5.0
>Reporter: Raul Gutierrez Segales
>Assignee: Michael Han
>Priority: Blocker
> Fix For: 3.5.3
>
> Attachments: ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, 
> ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, 
> ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch
>
>
> ZOOKEEPER-107 introduces reconfiguration support via the reconfig() call. We 
> should, at the very least, ensure that only the Admin can reconfigure a 
> cluster. Perhaps restricting access to /zookeeper/config as well, though this 
> is debatable. Surely one could ensure Admin only access via an ACL, but that 
> would leave everyone who doesn't use ACLs unprotected. We could also force a 
> default ACL to make it a bit more consistent (maybe).
> Finally, making reconfig() only available to Admins means they have to run 
> with zookeeper.DigestAuthenticationProvider.superDigest (which I am not sure 
> if everyone does, or how would it work with other authentication providers). 
> Review board https://reviews.apache.org/r/51546/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 51546: ZOOKEEPER-2014: Only admin should be allowed to reconfig a cluster

2016-09-28 Thread Michael Han 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51546/
---

(Updated Sept. 28, 2016, 11:39 p.m.)


Review request for zookeeper, fpj, Patrick Hunt, Raul Gutierrez Segales, and 
Alexander Shraer.


Changes
---

Update patch that includes C client side test update.


Bugs: ZOOKEEPER-2014
https://issues.apache.org/jira/browse/ZOOKEEPER-2014


Repository: zookeeper-git


Description
---

Address various security concerns around reconfig feature (ZOOKEEPER-2014) to 
unblock 3.5.3 release.


Diffs (updated)
-

  build.xml 96c9ef8 
  src/c/include/zookeeper.h 18a203d 
  src/c/tests/TestReconfigServer.cc 6a429ac 
  src/c/tests/ZooKeeperQuorumServer.h aa8b7cc 
  src/c/tests/ZooKeeperQuorumServer.cc 23392cd 
  src/docs/src/documentation/content/xdocs/zookeeperAdmin.xml f729095 
  src/docs/src/documentation/content/xdocs/zookeeperReconfig.xml 7168a01 
  src/java/main/org/apache/zookeeper/ClientCnxn.java 12dd51c 
  src/java/main/org/apache/zookeeper/KeeperException.java a05f1ab 
  src/java/main/org/apache/zookeeper/ZooKeeper.java f2ec3d7 
  src/java/main/org/apache/zookeeper/ZooKeeperMain.java 25d61a4 
  src/java/main/org/apache/zookeeper/admin/ZooKeeperAdmin.java PRE-CREATION 
  src/java/main/org/apache/zookeeper/cli/CliCommand.java 3d0a90b 
  src/java/main/org/apache/zookeeper/cli/ReconfigCommand.java deb7914 
  src/java/main/org/apache/zookeeper/server/DataTree.java 06c80d3 
  src/java/main/org/apache/zookeeper/server/PrepRequestProcessor.java 33f638d 
  src/java/main/org/apache/zookeeper/server/quorum/QuorumPeerConfig.java 
e772fa8 
  src/java/test/org/apache/zookeeper/server/DataTreeTest.java d726643 
  src/java/test/org/apache/zookeeper/server/quorum/ReconfigBackupTest.java 
241af52 
  
src/java/test/org/apache/zookeeper/server/quorum/ReconfigDuringLeaderSyncTest.java
 301837d 
  
src/java/test/org/apache/zookeeper/server/quorum/ReconfigFailureCasesTest.java 
e7147b3 
  src/java/test/org/apache/zookeeper/server/quorum/ReconfigLegacyTest.java 
ee9f2e2 
  src/java/test/org/apache/zookeeper/server/quorum/StandaloneDisabledTest.java 
1f6ce1f 
  src/java/test/org/apache/zookeeper/test/ACLTest.java 9920fc4 
  src/java/test/org/apache/zookeeper/test/ReconfigExceptionTest.java 
PRE-CREATION 
  src/java/test/org/apache/zookeeper/test/ReconfigTest.java 248a754 
  src/java/test/org/apache/zookeeper/test/StandaloneTest.java 5c95280 

Diff: https://reviews.apache.org/r/51546/diff/


Testing
---

Added new Java unit tests that cover various exception cases of using reconfig 
API without proper set up. 
All existing tests (Java and C) passed under stress tests (minors those known 
flaky tests.).
Manuelly tested reconfig commands using ZooKeeper command line tool.


Thanks,

Michael Han

[jira] [Updated] (ZOOKEEPER-2014) Only admin should be allowed to reconfig a cluster

2016-09-28 Thread Michael Han (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Han updated ZOOKEEPER-2014:
---
Attachment: ZOOKEEPER-2014.patch

Update patch to include C client work:
* Removed skipACL, set up servers with super user digest auth like Java tests 
did.
* Updated C client tests, add one specific test case to cover various reconfig 
failure cases caused by failure of auth.
* Misc update for first review feedback related to method signature and so on.

> Only admin should be allowed to reconfig a cluster
> --
>
> Key: ZOOKEEPER-2014
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2014
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.5.0
>Reporter: Raul Gutierrez Segales
>Assignee: Michael Han
>Priority: Blocker
> Fix For: 3.5.3
>
> Attachments: ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, 
> ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, 
> ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch
>
>
> ZOOKEEPER-107 introduces reconfiguration support via the reconfig() call. We 
> should, at the very least, ensure that only the Admin can reconfigure a 
> cluster. Perhaps restricting access to /zookeeper/config as well, though this 
> is debatable. Surely one could ensure Admin only access via an ACL, but that 
> would leave everyone who doesn't use ACLs unprotected. We could also force a 
> default ACL to make it a bit more consistent (maybe).
> Finally, making reconfig() only available to Admins means they have to run 
> with zookeeper.DigestAuthenticationProvider.superDigest (which I am not sure 
> if everyone does, or how would it work with other authentication providers). 
> Review board https://reviews.apache.org/r/51546/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

ZooKeeper-trunk-jdk8 - Build # 764 - Still Failing

2016-09-28 Thread Apache Jenkins Server 
See https://builds.apache.org/job/ZooKeeper-trunk-jdk8/764/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 463709 lines...]
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2016-09-28 12:10:59,523 [myid:] - INFO  [ProcessThread(sid:0 
cport:16852)::PrepRequestProcessor@647] - Processed session termination for 
sessionid: 0x10003b4f746
[junit] 2016-09-28 12:10:59,524 [myid:] - WARN  [New I/O worker 
#1811:NettyServerCnxnFactory$CnxnChannelHandler@142] - Exception caught [id: 
0x142905be, /127.0.0.1:42792 :> /127.0.0.1:16852] EXCEPTION: 
java.nio.channels.ClosedChannelException
[junit] java.nio.channels.ClosedChannelException
[junit] at 
sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:270)
[junit] at 
sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:461)
[junit] at 
org.jboss.netty.channel.socket.nio.SocketSendBufferPool$UnpooledSendBuffer.transferTo(SocketSendBufferPool.java:203)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.write0(AbstractNioWorker.java:201)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.writeFromTaskLoop(AbstractNioWorker.java:151)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioChannel$WriteTask.run(AbstractNioChannel.java:315)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
[junit] at 
org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
[junit] at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit] at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit] 2016-09-28 12:10:59,524 [myid:] - INFO  
[SyncThread:0:MBeanRegistry@128] - Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port16852,name1=Connections,name2=127.0.0.1,name3=0x10003b4f746]
[junit] 2016-09-28 12:10:59,624 [myid:] - INFO  [main:ZooKeeper@1313] - 
Session: 0x10003b4f746 closed
[junit] 2016-09-28 12:10:59,624 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x10003b4f746
[junit] 2016-09-28 12:10:59,625 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 52941
[junit] 2016-09-28 12:10:59,625 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 466
[junit] 2016-09-28 12:10:59,625 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD 
testWatcherAutoResetWithLocal
[junit] 2016-09-28 12:10:59,625 [myid:] - INFO  [main:ClientBase@543] - 
tearDown starting
[junit] 2016-09-28 12:10:59,625 [myid:] - INFO  [main:ClientBase@513] - 
STOPPING server
[junit] 2016-09-28 12:10:59,625 [myid:] - INFO  
[main:NettyServerCnxnFactory@464] - shutdown called 0.0.0.0/0.0.0.0:16852
[junit] 2016-09-28 12:10:59,627 [myid:] - INFO  [main:ZooKeeperServer@529] 
- shutting down
[junit] 2016-09-28 12:10:59,627 [myid:] - ERROR [main:ZooKeeperServer@501] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2016-09-28 12:10:59,627 [myid:] - INFO  
[main:SessionTrackerImpl@232] - Shutting down
[junit] 2016-09-28 12:10:59,627 [myid:] - INFO  
[main:PrepRequestProcessor@965] - Shutting down
[junit] 2016-09-28 12:10:59,627 [myid:] - INFO  
[main:SyncRequestProcessor@191] - Shutting down
[junit] 2016-09-28 12:10:59,627 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
[junit] 2016-09-28 12:10:59,627 [myid:] - INFO  [ProcessThread(sid:0 
cport:16852)::PrepRequestProcessor@154] - PrepRequestProcessor exited loop!
[junit] 2016-09-28 12:10:59,627 [myid:] - INFO  
[main:FinalRequestProcessor@479] - shutdown of request processor complete
[junit] 2016-09-28 12:10:59,628 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port16852,name1=InMemoryDataTree]
[junit] 2016-09-28

ZooKeeper_branch35_openjdk7 - Build # 245 - Still Failing

2016-09-28 Thread Apache Jenkins Server 
See https://builds.apache.org/job/ZooKeeper_branch35_openjdk7/245/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 469734 lines...]
[junit] 2016-09-28 11:10:16,133 [myid:127.0.0.1:19376] - INFO  
[main-SendThread(127.0.0.1:19376):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19376. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2016-09-28 11:10:16,133 [myid:127.0.0.1:19376] - INFO  
[main-SendThread(127.0.0.1:19376):ClientCnxn$SendThread@948] - Socket 
connection established, initiating session, client: /127.0.0.1:41225, server: 
127.0.0.1/127.0.0.1:19376
[junit] 2016-09-28 11:10:16,133 [myid:] - INFO  
[NIOServerCxnFactory.AcceptThread:/127.0.0.1:19376:NIOServerCnxnFactory$AcceptThread@296]
 - Accepted socket connection from /127.0.0.1:41225
[junit] 2016-09-28 11:10:16,134 [myid:] - WARN  
[NIOWorkerThread-22:NIOServerCnxn@369] - Exception causing close of session 
0x0: ZooKeeperServer not running
[junit] 2016-09-28 11:10:16,134 [myid:] - INFO  
[NIOWorkerThread-22:NIOServerCnxn@607] - Closed socket connection for client 
/127.0.0.1:41225 (no session established for client)
[junit] 2016-09-28 11:10:16,134 [myid:127.0.0.1:19376] - INFO  
[main-SendThread(127.0.0.1:19376):ClientCnxn$SendThread@1231] - Unable to read 
additional data from server sessionid 0x0, likely server has closed socket, 
closing socket connection and attempting reconnect
[junit] 2016-09-28 11:10:16,651 [myid:127.0.0.1:19355] - INFO  
[main-SendThread(127.0.0.1:19355):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19355. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2016-09-28 11:10:16,652 [myid:127.0.0.1:19355] - WARN  
[main-SendThread(127.0.0.1:19355):ClientCnxn$SendThread@1235] - Session 
0x30103f3bc2e for server 127.0.0.1/127.0.0.1:19355, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2016-09-28 11:10:17,102 [myid:127.0.0.1:19352] - INFO  
[main-SendThread(127.0.0.1:19352):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19352. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2016-09-28 11:10:17,103 [myid:127.0.0.1:19352] - WARN  
[main-SendThread(127.0.0.1:19352):ClientCnxn$SendThread@1235] - Session 
0x20103f3bd4e for server 127.0.0.1/127.0.0.1:19352, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2016-09-28 11:10:17,487 [myid:127.0.0.1:19376] - INFO  
[main-SendThread(127.0.0.1:19376):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19376. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2016-09-28 11:10:17,487 [myid:127.0.0.1:19376] - INFO  
[main-SendThread(127.0.0.1:19376):ClientCnxn$SendThread@948] - Socket 
connection established, initiating session, client: /127.0.0.1:41272, server: 
127.0.0.1/127.0.0.1:19376
[junit] 2016-09-28 11:10:17,487 [myid:] - INFO  
[NIOServerCxnFactory.AcceptThread:/127.0.0.1:19376:NIOServerCnxnFactory$AcceptThread@296]
 - Accepted socket connection from /127.0.0.1:41272
[junit] 2016-09-28 11:10:17,488 [myid:] - WARN  
[NIOWorkerThread-23:NIOServerCnxn@369] - Exception causing close of session 
0x0: ZooKeeperServer not running
[junit] 2016-09-28 11:10:17,488 [myid:] - INFO  
[NIOWorkerThread-23:NIOServerCnxn@607] - Closed socket connection for client 
/127.0.0.1:41272 (no session established for client)
[junit] 2016-09-28 11:10:17,488 [myid:127.0.0.1:19376] - INFO  
[main-SendThread(127.0.0.1:19376):ClientCnxn$SendThread@1231] - Unable to read 
additional data from server sessionid 0x0, likely server has closed socket, 
closing socket connection and attempting reconnect
[junit] 2016-09-28 11:10:17,643 [myid:127.0.0.1:19349] - INFO  
[main-SendThread(127.0.0.1:19349):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19349. Will not

Re: ZOOKEEPER-2169

2016-09-28 Thread Jordan Zimmerman 
FYI - I’ve applied the last remaining request from Review Board and moved the 
patch to Github:

https://github.com/apache/zookeeper/pull/82

-Jordan

> On Aug 30, 2016, at 7:27 PM, Jordan Zimmerman  
> wrote:
> 
>> As I understand TTL nodes, you want to have them when you don't want to have 
>> sessions
> 
> That’s not the only use case. For example, here at Elasticsearch we have a 
> need for a semi-persistent node. It’s a node that lasts for about an hour. 
> Whether or not there is a session is irrelevant. I ended up adding a recipe 
> to Curator for it but it would be much nicer to have direct support in ZK for 
> this. Again, I think you’re conflating the session issue with TTL nodes. They 
> are different topics.
> 
> -Jordan
> 
>> On Aug 30, 2016, at 12:23 PM, Flavio Junqueira  wrote:
>> 
>> As I understand TTL nodes, you want to have them when you don't want to have 
>> sessions. I find it odd that you still need to create a session when you 
>> choose to use TTL nodes to avoid sessions. It is correct that you can create 
>> session/create TTL/close session in this order, but again, if you're trying 
>> to avoid sessions, then it doesn't seem to be very appealing to use TTL 
>> nodes this way.
>> 
>> In any case, I need to go through the e-mail thread that Camille pointed 
>> out. There is possibly some insight there that I'm missing.
>> 
>> -Flavio
>> 
>>> On 30 Aug 2016, at 14:21, Jordan Zimmerman  
>>> wrote:
>>> 
>>> Yes, you need a session to create the TTL node. I believe discussion about 
>>> needing a session to create the node is beyond the scope of this issue and 
>>> should be addressed by a new Jira issue. It doesn’t affect the utility of 
>>> TTL nodes that you must first have a ZK session. Users who no longer want a 
>>> session can merely close the ZK handle after creating the TTL node. 
>>> 
>>> -Jordan
>>> 
 On Aug 30, 2016, at 7:41 AM, Flavio Junqueira  wrote:
 
 
> On 29 Aug 2016, at 19:51, Jordan Zimmerman  
> wrote:
> 
>> On the server side, we already have a mechanism to expire sessions, do 
>> we a separate scheme to expire TTL nodes or can we use the same 
>> mechanism? Does it make sense to consider a TTL node as a degenerate 
>> case of a session in which I have a single ephemeral node? My 
>> recollection is that it currently uses the container manager instead.
> 
> The TTL implementation I did takes advantage of the Container node 
> feature. A TTL node is a variation of a container node. It doesn’t 
> require a session (like any persistent node). 
> 
 
 It doesn't require a session, but the client has no way to create such a 
 TTL node without creating a session first, right? You need the zk handle, 
 which has a session associated, to create a TTL node as I understand it.
 
 -Flavio
 
>>> 
>>> 
>> 
>

[jira] [Commented] (ZOOKEEPER-2169) Enable creation of nodes with TTLs

2016-09-28 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528906#comment-15528906
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2169:
---

GitHub user Randgalt opened a pull request:

https://github.com/apache/zookeeper/pull/82

[ZOOKEEPER-2169] Enable creation of nodes with TTLs

This patch takes advantage of 3.5's container support. Most of the work 
needed to support TTLs is there already.

In order not to break on-disk and protocol compatibility the ephemeralOwner 
is yet-again overloaded to have special meaning.
New opcodes and transaction records had to be added in a similar manner to 
Containers

NOTE: This patch was originally in Review Board and was moved to Github for 
ease/convenience. RB link:
https://reviews.apache.org/r/46983/#comment214810

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Randgalt/zookeeper ZOOKEEPER-2169

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/82.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #82


commit 9cb973c7ff77b9a12c2fb74d05da38b79ab34c07
Author: randgalt 
Date:   2016-09-28T08:32:50Z

This patch takes advantage of 3.5's container support. Most of the work 
needed to support TTLs is there already.
In order not to break on-disk and protocol compatibility the ephemeralOwner 
is yet-again overloaded to have special meaning.
New opcodes and transaction records had to be added in a similar manner to 
Containers

NOTE: This patch was originally in Review Board and was moved to Github for 
ease/convenience. RB link:
https://reviews.apache.org/r/46983/#comment214810




> Enable creation of nodes with TTLs
> --
>
> Key: ZOOKEEPER-2169
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2169
> Project: ZooKeeper
>  Issue Type: New Feature
>  Components: c client, java client, jute, server
>Affects Versions: 3.6.0
>Reporter: Camille Fournier
>Assignee: Jordan Zimmerman
> Fix For: 3.6.0
>
> Attachments: ZOOKEEPER-2169-2.patch, ZOOKEEPER-2169-3.patch, 
> ZOOKEEPER-2169-4.patch, ZOOKEEPER-2169-5.patch, ZOOKEEPER-2169.patch
>
>
> As a user, I would like to be able to create a node that is NOT tied to a 
> session but that WILL expire automatically if action is not taken by some 
> client within a time window.
> I propose this to enable clients interacting with ZK via http or other "thin 
> clients" to create ephemeral-like nodes.
> Some ideas for the design, up for discussion:
> The node should support all normal ZK node operations including ACLs, 
> sequential key generation, etc, however, it should not support the ephemeral 
> flag. The node will be created with a TTL that is updated via a refresh 
> operation. 
> The ZK quorum will watch this node similarly to the way that it watches for 
> session liveness; if the node is not refreshed within the TTL, it will expire.
> QUESTIONS:
> 1) Should we let the refresh operation set the TTL to a different base value?
> 2) If so, should the setting of the TTL to a new base value cause a watch to 
> fire?
> 3) Do we want to allow these nodes to have children or prevent this similar 
> to ephemeral nodes?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] zookeeper pull request #82: [ZOOKEEPER-2169] Enable creation of nodes with T...

2016-09-28 Thread Randgalt 
GitHub user Randgalt opened a pull request:

https://github.com/apache/zookeeper/pull/82

[ZOOKEEPER-2169] Enable creation of nodes with TTLs

This patch takes advantage of 3.5's container support. Most of the work 
needed to support TTLs is there already.

In order not to break on-disk and protocol compatibility the ephemeralOwner 
is yet-again overloaded to have special meaning.
New opcodes and transaction records had to be added in a similar manner to 
Containers

NOTE: This patch was originally in Review Board and was moved to Github for 
ease/convenience. RB link:
https://reviews.apache.org/r/46983/#comment214810

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Randgalt/zookeeper ZOOKEEPER-2169

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/82.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #82


commit 9cb973c7ff77b9a12c2fb74d05da38b79ab34c07
Author: randgalt 
Date:   2016-09-28T08:32:50Z

This patch takes advantage of 3.5's container support. Most of the work 
needed to support TTLs is there already.
In order not to break on-disk and protocol compatibility the ephemeralOwner 
is yet-again overloaded to have special meaning.
New opcodes and transaction records had to be added in a similar manner to 
Containers

NOTE: This patch was originally in Review Board and was moved to Github for 
ease/convenience. RB link:
https://reviews.apache.org/r/46983/#comment214810




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (ZOOKEEPER-2014) Only admin should be allowed to reconfig a cluster

2016-09-28 Thread Flavio Junqueira (JIRA) 

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528898#comment-15528898
 ] 

Flavio Junqueira commented on ZOOKEEPER-2014:
-

I'll wait until you have the C client changes to make another pass, unless you 
need feedback on some specific parts to make progress. [~hanm]

> Only admin should be allowed to reconfig a cluster
> --
>
> Key: ZOOKEEPER-2014
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2014
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.5.0
>Reporter: Raul Gutierrez Segales
>Assignee: Michael Han
>Priority: Blocker
> Fix For: 3.5.3
>
> Attachments: ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, 
> ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, ZOOKEEPER-2014.patch, 
> ZOOKEEPER-2014.patch
>
>
> ZOOKEEPER-107 introduces reconfiguration support via the reconfig() call. We 
> should, at the very least, ensure that only the Admin can reconfigure a 
> cluster. Perhaps restricting access to /zookeeper/config as well, though this 
> is debatable. Surely one could ensure Admin only access via an ACL, but that 
> would leave everyone who doesn't use ACLs unprotected. We could also force a 
> default ACL to make it a bit more consistent (maybe).
> Finally, making reconfig() only available to Admins means they have to run 
> with zookeeper.DigestAuthenticationProvider.superDigest (which I am not sure 
> if everyone does, or how would it work with other authentication providers). 
> Review board https://reviews.apache.org/r/51546/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

ZooKeeper-trunk-solaris - Build # 1328 - Still Failing

2016-09-28 Thread Apache Jenkins Server 
See https://builds.apache.org/job/ZooKeeper-trunk-solaris/1328/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 441541 lines...]
[junit] 2016-09-28 08:28:59,738 [myid:] - INFO  [main:ClientBase@386] - 
CREATING server instance 127.0.0.1:11222
[junit] 2016-09-28 08:28:59,738 [myid:] - INFO  
[main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with 10s 
sessionless connection timeout, 2 selector thread(s), 16 worker threads, and 64 
kB direct buffers.
[junit] 2016-09-28 08:28:59,739 [myid:] - INFO  
[main:NIOServerCnxnFactory@686] - binding to port 0.0.0.0/0.0.0.0:11222
[junit] 2016-09-28 08:28:59,739 [myid:] - INFO  [main:ClientBase@361] - 
STARTING server instance 127.0.0.1:11222
[junit] 2016-09-28 08:28:59,740 [myid:] - INFO  [main:ZooKeeperServer@889] 
- minSessionTimeout set to 6000
[junit] 2016-09-28 08:28:59,740 [myid:] - INFO  [main:ZooKeeperServer@898] 
- maxSessionTimeout set to 6
[junit] 2016-09-28 08:28:59,740 [myid:] - INFO  [main:ZooKeeperServer@159] 
- Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 
6 datadir 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/build/test/tmp/test5512352702224147518.junit.dir/version-2
 snapdir 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/build/test/tmp/test5512352702224147518.junit.dir/version-2
[junit] 2016-09-28 08:28:59,741 [myid:] - INFO  [main:FileSnap@83] - 
Reading snapshot 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/build/test/tmp/test5512352702224147518.junit.dir/version-2/snapshot.b
[junit] 2016-09-28 08:28:59,743 [myid:] - INFO  [main:FileTxnSnapLog@306] - 
Snapshotting: 0xb to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/build/test/tmp/test5512352702224147518.junit.dir/version-2/snapshot.b
[junit] 2016-09-28 08:28:59,744 [myid:] - ERROR [main:ZooKeeperServer@501] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2016-09-28 08:28:59,744 [myid:] - INFO  
[main:FourLetterWordMain@85] - connecting to 127.0.0.1 11222
[junit] 2016-09-28 08:28:59,745 [myid:] - INFO  
[NIOServerCxnFactory.AcceptThread:0.0.0.0/0.0.0.0:11222:NIOServerCnxnFactory$AcceptThread@296]
 - Accepted socket connection from /127.0.0.1:43727
[junit] 2016-09-28 08:28:59,745 [myid:] - INFO  
[NIOWorkerThread-1:NIOServerCnxn@485] - Processing stat command from 
/127.0.0.1:43727
[junit] 2016-09-28 08:28:59,745 [myid:] - INFO  
[NIOWorkerThread-1:StatCommand@49] - Stat command output
[junit] 2016-09-28 08:28:59,746 [myid:] - INFO  
[NIOWorkerThread-1:NIOServerCnxn@607] - Closed socket connection for client 
/127.0.0.1:43727 (no session established for client)
[junit] 2016-09-28 08:28:59,746 [myid:] - INFO  [main:JMXEnv@228] - 
ensureParent:[InMemoryDataTree, StandaloneServer_port]
[junit] 2016-09-28 08:28:59,747 [myid:] - INFO  [main:JMXEnv@245] - 
expect:InMemoryDataTree
[junit] 2016-09-28 08:28:59,747 [myid:] - INFO  [main:JMXEnv@249] - 
found:InMemoryDataTree 
org.apache.ZooKeeperService:name0=StandaloneServer_port11222,name1=InMemoryDataTree
[junit] 2016-09-28 08:28:59,747 [myid:] - INFO  [main:JMXEnv@245] - 
expect:StandaloneServer_port
[junit] 2016-09-28 08:28:59,748 [myid:] - INFO  [main:JMXEnv@249] - 
found:StandaloneServer_port 
org.apache.ZooKeeperService:name0=StandaloneServer_port11222
[junit] 2016-09-28 08:28:59,748 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 17771
[junit] 2016-09-28 08:28:59,748 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 24
[junit] 2016-09-28 08:28:59,748 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD 
testQuota
[junit] 2016-09-28 08:28:59,748 [myid:] - INFO  [main:ClientBase@543] - 
tearDown starting
[junit] 2016-09-28 08:28:59,822 [myid:] - INFO  [main:ZooKeeper@1313] - 
Session: 0x123ece46871 closed
[junit] 2016-09-28 08:28:59,822 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x123ece46871
[junit] 2016-09-28 08:28:59,822 [myid:] - INFO  [main:ClientBase@513] - 
STOPPING server
[junit] 2016-09-28 08:28:59,822 [myid:] - INFO  
[ConnnectionExpirer:NIOServerCnxnFactory$ConnectionExpirerThread@583] - 
ConnnectionExpirerThread interrupted
[junit] 2016-09-28 08:28:59,822 [myid:] - INFO  
[NIOServerCxnFactory.SelectorThread-1:NIOServerCnxnFactory$SelectorThread@420] 
- selector thread exitted run method
[junit] 2016-09-28 08:28:59,822 [myid:] - INFO  
[NIOServerCxnFactory.SelectorThread-0:NIOServerCnxnFactory$SelectorThread@420] 
-