[jira] [Commented] (ZOOKEEPER-2994) Tool required to recover log and snapshot entries with CRC errors

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453500#comment-16453500
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2994:
---

Github user anmolnar closed the pull request at:

https://github.com/apache/zookeeper/pull/508


> Tool required to recover log and snapshot entries with CRC errors
> -
>
> Key: ZOOKEEPER-2994
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2994
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.5.4, 3.6.0, 3.4.13
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 3.5.4, 3.6.0, 3.4.13
>
>
> In the even that the zookeeper transaction log or snapshot become corrupted 
> and fail CRC checks (preventing startup) we should have a mechanism to get 
> the cluster running again.
> Previously we achieved this by loading the broken transaction log with a 
> modified version of ZK with disabled CRC check and forced it to snapshot.
> It'd very handy to have a tool which can do this for us. LogFormatter and 
> SnapshotFormatter have already been designed to dump log and snapshot files, 
> it'd be nice to extend their functionality and add ability for such recovery.
> It has proven that once you end up with the corrupt txn log there is no way 
> to recover except manually modifying the crc check. That's basically why the 
> tool is needed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper pull request #508: ZOOKEEPER-2994 Tool required to recover log and...

2018-04-25 Thread anmolnar
Github user anmolnar closed the pull request at:

https://github.com/apache/zookeeper/pull/508


---


[jira] [Commented] (ZOOKEEPER-3007) Potential NPE in ReferenceCountedACLCache#deserialize

2018-04-25 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453380#comment-16453380
 ] 

Hadoop QA commented on ZOOKEEPER-3007:
--

-1 overall.  GitHub Pull Request  Build
  

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 3.0.1) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1632//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1632//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1632//console

This message is automatically generated.

> Potential NPE in ReferenceCountedACLCache#deserialize 
> --
>
> Key: ZOOKEEPER-3007
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3007
> Project: ZooKeeper
>  Issue Type: Bug
>Affects Versions: 3.6.0
>Reporter: lujie
>Priority: Major
>
> Inspired by ZK-3006 , I develop a simple static analysis tool to find other 
> Potential NPE like ZK-3006. This bug is found by this tool ,and I have 
> carefully studied it.  But i am a newbie at here so i may be wrong, hope 
> someone could confirm it and help me improve this tool.
> h3. Bug describtion:
> callee BinaryInputArchive#startVector will return null:
> {code:java}
> // code placeholder
> public Index startVector(String tag) throws IOException {
> int len = readInt(tag);
>  if (len == -1) {
>  return null;
> }
> {code}
> and caller ReferenceCountedACLCache#deserialize  call it without null check
> {code:java}
> // code placeholder
> Index j = ia.startVector("acls");
> while (!j.done()) {
>   ACL acl = new ACL();
>   acl.deserialize(ia, "acl");
> }{code}
> but all the other 14 caller of BinaryInputArchive#startVector performs null 
> checker like:
> {code:java}
> // code placeholder
> Index vidx1 = a_.startVector("acl");
>   if (vidx1!= null)
>  for (; !vidx1.done(); vidx1.incr()){
>  .
> }
>}
> }
> {code}
> so i think we also need add null check in caller 
> ReferenceCountedACLCache#deserialize  just like other 14 caller
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Failed: ZOOKEEPER- PreCommit Build #1632

2018-04-25 Thread Apache Jenkins Server
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1632/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 75.44 MB...]
 [exec] 
 [exec] +1 javadoc.  The javadoc tool did not generate any warning 
messages.
 [exec] 
 [exec] +1 javac.  The applied patch does not increase the total number 
of javac compiler warnings.
 [exec] 
 [exec] +1 findbugs.  The patch does not introduce any new Findbugs 
(version 3.0.1) warnings.
 [exec] 
 [exec] +1 release audit.  The applied patch does not increase the 
total number of release audit warnings.
 [exec] 
 [exec] -1 core tests.  The patch failed core unit tests.
 [exec] 
 [exec] +1 contrib tests.  The patch passed contrib unit tests.
 [exec] 
 [exec] Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1632//testReport/
 [exec] Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1632//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
 [exec] Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1632//console
 [exec] 
 [exec] This message is automatically generated.
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Adding comment to Jira.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] Comment with id 16453380 added to ZOOKEEPER-3007.
 [exec] Session logged out. Session was 
JSESSIONID=CB191E8A2A8BCC7F983469C180883F8B.
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Finished build.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] mv: 
'/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess'
 and 
'/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess'
 are the same file

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml:1737:
 exec returned: 1

Total time: 15 minutes 50 seconds
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8
Recording test results
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8
[description-setter] Description set: ZOOKEEPER-3007
Putting comment on the pull request
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8



###
## FAILED TESTS (if any) 
##
3 tests failed.
FAILED:  org.apache.zookeeper.test.AsyncHammerTest.testHammer

Error Message:
null

Stack Trace:
junit.framework.AssertionFailedError
at 
org.apache.zookeeper.test.AsyncHammerTest.testHammer(AsyncHammerTest.java:185)
at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange

Error Message:
KeeperErrorCode = Session expired

Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired
at org.apache.zookeeper.KeeperException.create(KeeperException.java:134)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at 
org.apache.zookeeper.admin.ZooKeeperAdmin.reconfigure(ZooKeeperAdmin.java:187)
at 
org.apache.zookeeper.admin.ZooKeeperAdmin.reconfigure(ZooKeeperAdmin.java:204)
at org.apache.zookeeper.test.ReconfigTest.reconfig(ReconfigTest.java:84)
at 
org.apache.zookeeper.test.ReconfigTest.testQuorumSystemChange(ReconfigTest.java:870)
at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)


FAILED:  org.apache.zookeeper.test.SSLTest.testSecureQuorumServer

Error Message:
waiti

[jira] [Commented] (ZOOKEEPER-3008) Potential NPE in SaslQuorumAuthLearner#authenticate and SaslQuorumAuthServer#authenticate

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453346#comment-16453346
 ] 

ASF GitHub Bot commented on ZOOKEEPER-3008:
---

Github user lujiefsi commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/496#discussion_r184251756
  
--- Diff: 
src/java/main/org/apache/zookeeper/server/quorum/auth/SaslQuorumAuthLearner.java
 ---
@@ -94,7 +94,10 @@ public void authenticate(Socket sock, String hostName) 
throws IOException {
 principalConfig,
 QuorumAuth.QUORUM_SERVER_PROTOCOL_NAME,
 QuorumAuth.QUORUM_SERVER_SASL_DIGEST, LOG, 
"QuorumLearner");
-
+if (sc == null) {
--- End diff --

For #1:
Follower#77,Observer#69,QuorumCnxManager#333 all have same patern:
`try { //root caller } catch (IOException e) {//handler code}`
#2 and #3, @brettKK .


> Potential NPE in SaslQuorumAuthLearner#authenticate and 
> SaslQuorumAuthServer#authenticate
> -
>
> Key: ZOOKEEPER-3008
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3008
> Project: ZooKeeper
>  Issue Type: Bug
>Affects Versions: 3.6.0
>Reporter: lujie
>Priority: Major
>
> Inspired by ZK-3006 , I develop a simple static analysis tool to find other 
> Potential NPE like ZK-3006. This bug is found by this tool ,and I have 
> carefully studied it.  But i am a newbie at here so i may be wrong, hope 
> someone could confirm it and help me improve this tool.
> h2. Bug description:
> callee :SecurityUtils#createSaslClient will return null while encounter 
> exception
> {code:java}
> // code placeholder
> catch (Exception e) {
>   LOG.error("Exception while trying to create SASL client", e);
>   return null;
> }
> {code}
> but its caller has no null check just like:
> {code:java}
> // code placeholder
> sc = SecurityUtils.createSaslClient();
> if (sc.hasInitialResponse()) {
>responseToken = createSaslToken(new byte[0], sc, learnerLogin);
> }
> {code}
> I think we should add null check in caller while callee return null



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper pull request #496: ZOOKEEPER-3008: Potential NPE in SaslQuorumAuth...

2018-04-25 Thread lujiefsi
Github user lujiefsi commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/496#discussion_r184251756
  
--- Diff: 
src/java/main/org/apache/zookeeper/server/quorum/auth/SaslQuorumAuthLearner.java
 ---
@@ -94,7 +94,10 @@ public void authenticate(Socket sock, String hostName) 
throws IOException {
 principalConfig,
 QuorumAuth.QUORUM_SERVER_PROTOCOL_NAME,
 QuorumAuth.QUORUM_SERVER_SASL_DIGEST, LOG, 
"QuorumLearner");
-
+if (sc == null) {
--- End diff --

For #1:
Follower#77,Observer#69,QuorumCnxManager#333 all have same patern:
`try { //root caller } catch (IOException e) {//handler code}`
#2 and #3, @brettKK .


---


[jira] [Commented] (ZOOKEEPER-2955) Enable Clover code coverage report

2018-04-25 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453213#comment-16453213
 ] 

Hudson commented on ZOOKEEPER-2955:
---

SUCCESS: Integrated in Jenkins build ZooKeeper-trunk #3818 (See 
[https://builds.apache.org/job/ZooKeeper-trunk/3818/])
ZOOKEEPER-2955: Enable Clover code coverage report (phunt: rev 
79627c6e07a901150709896f7c22597164badfc5)
* (edit) build.xml
* (edit) ivy.xml


> Enable Clover code coverage report
> --
>
> Key: ZOOKEEPER-2955
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2955
> Project: ZooKeeper
>  Issue Type: Test
>  Components: tests
>Affects Versions: 3.5.3, 3.4.11
>Reporter: Mark Fenes
>Assignee: Mark Fenes
>Priority: Major
>
> We have limited code coverage support in ZK. Clover for Java was running in 
> the past but was turned off. 
> Enable Clover code coverage report to make us more confident on the quality, 
> stability and compatibility of future ZK releases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ZOOKEEPER-2955) Enable Clover code coverage report

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453183#comment-16453183
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2955:
---

Github user phunt commented on the issue:

https://github.com/apache/zookeeper/pull/444
  
Does this need to be updated to reflect #443 ?


> Enable Clover code coverage report
> --
>
> Key: ZOOKEEPER-2955
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2955
> Project: ZooKeeper
>  Issue Type: Test
>  Components: tests
>Affects Versions: 3.5.3, 3.4.11
>Reporter: Mark Fenes
>Assignee: Mark Fenes
>Priority: Major
>
> We have limited code coverage support in ZK. Clover for Java was running in 
> the past but was turned off. 
> Enable Clover code coverage report to make us more confident on the quality, 
> stability and compatibility of future ZK releases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ZOOKEEPER-2955) Enable Clover code coverage report

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453185#comment-16453185
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2955:
---

Github user phunt commented on the issue:

https://github.com/apache/zookeeper/pull/445
  
Does this need to be updated to reflect #443 ?


> Enable Clover code coverage report
> --
>
> Key: ZOOKEEPER-2955
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2955
> Project: ZooKeeper
>  Issue Type: Test
>  Components: tests
>Affects Versions: 3.5.3, 3.4.11
>Reporter: Mark Fenes
>Assignee: Mark Fenes
>Priority: Major
>
> We have limited code coverage support in ZK. Clover for Java was running in 
> the past but was turned off. 
> Enable Clover code coverage report to make us more confident on the quality, 
> stability and compatibility of future ZK releases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper issue #444: ZOOKEEPER-2955: Enable Clover code coverage report

2018-04-25 Thread phunt
Github user phunt commented on the issue:

https://github.com/apache/zookeeper/pull/444
  
Does this need to be updated to reflect #443 ?


---


[jira] [Commented] (ZOOKEEPER-2955) Enable Clover code coverage report

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453181#comment-16453181
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2955:
---

Github user phunt commented on the issue:

https://github.com/apache/zookeeper/pull/443
  
+1, lgtm. I've committed this to master. Thanks @mfenes !


> Enable Clover code coverage report
> --
>
> Key: ZOOKEEPER-2955
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2955
> Project: ZooKeeper
>  Issue Type: Test
>  Components: tests
>Affects Versions: 3.5.3, 3.4.11
>Reporter: Mark Fenes
>Assignee: Mark Fenes
>Priority: Major
>
> We have limited code coverage support in ZK. Clover for Java was running in 
> the past but was turned off. 
> Enable Clover code coverage report to make us more confident on the quality, 
> stability and compatibility of future ZK releases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper issue #445: ZOOKEEPER-2955: Enable Clover code coverage report

2018-04-25 Thread phunt
Github user phunt commented on the issue:

https://github.com/apache/zookeeper/pull/445
  
Does this need to be updated to reflect #443 ?


---


[GitHub] zookeeper issue #443: ZOOKEEPER-2955: Enable Clover code coverage report

2018-04-25 Thread phunt
Github user phunt commented on the issue:

https://github.com/apache/zookeeper/pull/443
  
+1, lgtm. I've committed this to master. Thanks @mfenes !


---


[jira] [Commented] (ZOOKEEPER-2955) Enable Clover code coverage report

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453180#comment-16453180
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2955:
---

Github user asfgit closed the pull request at:

https://github.com/apache/zookeeper/pull/443


> Enable Clover code coverage report
> --
>
> Key: ZOOKEEPER-2955
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2955
> Project: ZooKeeper
>  Issue Type: Test
>  Components: tests
>Affects Versions: 3.5.3, 3.4.11
>Reporter: Mark Fenes
>Assignee: Mark Fenes
>Priority: Major
>
> We have limited code coverage support in ZK. Clover for Java was running in 
> the past but was turned off. 
> Enable Clover code coverage report to make us more confident on the quality, 
> stability and compatibility of future ZK releases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper pull request #443: ZOOKEEPER-2955: Enable Clover code coverage rep...

2018-04-25 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/zookeeper/pull/443


---


[jira] [Commented] (ZOOKEEPER-3008) Potential NPE in SaslQuorumAuthLearner#authenticate and SaslQuorumAuthServer#authenticate

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453159#comment-16453159
 ] 

ASF GitHub Bot commented on ZOOKEEPER-3008:
---

Github user phunt commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/496#discussion_r184218439
  
--- Diff: 
src/java/main/org/apache/zookeeper/server/quorum/auth/SaslQuorumAuthLearner.java
 ---
@@ -94,7 +94,10 @@ public void authenticate(Socket sock, String hostName) 
throws IOException {
 principalConfig,
 QuorumAuth.QUORUM_SERVER_PROTOCOL_NAME,
 QuorumAuth.QUORUM_SERVER_SASL_DIGEST, LOG, 
"QuorumLearner");
-
+if (sc == null) {
--- End diff --

Same feedback as #495 

1) check the callers and see if it's handled properly. Likely it will be 
logged there as well. Verify/report.
2) No need to say exception in an exception. The text of LOG.error line 
seems like it would have been a good error string for the exception itself.
3) as previously noted, add a test.


> Potential NPE in SaslQuorumAuthLearner#authenticate and 
> SaslQuorumAuthServer#authenticate
> -
>
> Key: ZOOKEEPER-3008
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3008
> Project: ZooKeeper
>  Issue Type: Bug
>Affects Versions: 3.6.0
>Reporter: lujie
>Priority: Major
>
> Inspired by ZK-3006 , I develop a simple static analysis tool to find other 
> Potential NPE like ZK-3006. This bug is found by this tool ,and I have 
> carefully studied it.  But i am a newbie at here so i may be wrong, hope 
> someone could confirm it and help me improve this tool.
> h2. Bug description:
> callee :SecurityUtils#createSaslClient will return null while encounter 
> exception
> {code:java}
> // code placeholder
> catch (Exception e) {
>   LOG.error("Exception while trying to create SASL client", e);
>   return null;
> }
> {code}
> but its caller has no null check just like:
> {code:java}
> // code placeholder
> sc = SecurityUtils.createSaslClient();
> if (sc.hasInitialResponse()) {
>responseToken = createSaslToken(new byte[0], sc, learnerLogin);
> }
> {code}
> I think we should add null check in caller while callee return null



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper pull request #496: ZOOKEEPER-3008: Potential NPE in SaslQuorumAuth...

2018-04-25 Thread phunt
Github user phunt commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/496#discussion_r184218439
  
--- Diff: 
src/java/main/org/apache/zookeeper/server/quorum/auth/SaslQuorumAuthLearner.java
 ---
@@ -94,7 +94,10 @@ public void authenticate(Socket sock, String hostName) 
throws IOException {
 principalConfig,
 QuorumAuth.QUORUM_SERVER_PROTOCOL_NAME,
 QuorumAuth.QUORUM_SERVER_SASL_DIGEST, LOG, 
"QuorumLearner");
-
+if (sc == null) {
--- End diff --

Same feedback as #495 

1) check the callers and see if it's handled properly. Likely it will be 
logged there as well. Verify/report.
2) No need to say exception in an exception. The text of LOG.error line 
seems like it would have been a good error string for the exception itself.
3) as previously noted, add a test.


---


[jira] [Commented] (ZOOKEEPER-2994) Tool required to recover log and snapshot entries with CRC errors

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453143#comment-16453143
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2994:
---

Github user phunt commented on the issue:

https://github.com/apache/zookeeper/pull/508
  
@anmolnar you will have to close this manually as it's not master branch.



> Tool required to recover log and snapshot entries with CRC errors
> -
>
> Key: ZOOKEEPER-2994
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2994
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.5.4, 3.6.0, 3.4.13
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 3.5.4, 3.6.0, 3.4.13
>
>
> In the even that the zookeeper transaction log or snapshot become corrupted 
> and fail CRC checks (preventing startup) we should have a mechanism to get 
> the cluster running again.
> Previously we achieved this by loading the broken transaction log with a 
> modified version of ZK with disabled CRC check and forced it to snapshot.
> It'd very handy to have a tool which can do this for us. LogFormatter and 
> SnapshotFormatter have already been designed to dump log and snapshot files, 
> it'd be nice to extend their functionality and add ability for such recovery.
> It has proven that once you end up with the corrupt txn log there is no way 
> to recover except manually modifying the crc check. That's basically why the 
> tool is needed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper issue #508: ZOOKEEPER-2994 Tool required to recover log and snapsh...

2018-04-25 Thread phunt
Github user phunt commented on the issue:

https://github.com/apache/zookeeper/pull/508
  
@anmolnar you will have to close this manually as it's not master branch.



---


[jira] [Commented] (ZOOKEEPER-2994) Tool required to recover log and snapshot entries with CRC errors

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453134#comment-16453134
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2994:
---

Github user phunt commented on the issue:

https://github.com/apache/zookeeper/pull/508
  
+1. Great. Thanks @anmolnar . I've committed for 3.4.13.


> Tool required to recover log and snapshot entries with CRC errors
> -
>
> Key: ZOOKEEPER-2994
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2994
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.5.4, 3.6.0, 3.4.13
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 3.5.4, 3.6.0, 3.4.13
>
>
> In the even that the zookeeper transaction log or snapshot become corrupted 
> and fail CRC checks (preventing startup) we should have a mechanism to get 
> the cluster running again.
> Previously we achieved this by loading the broken transaction log with a 
> modified version of ZK with disabled CRC check and forced it to snapshot.
> It'd very handy to have a tool which can do this for us. LogFormatter and 
> SnapshotFormatter have already been designed to dump log and snapshot files, 
> it'd be nice to extend their functionality and add ability for such recovery.
> It has proven that once you end up with the corrupt txn log there is no way 
> to recover except manually modifying the crc check. That's basically why the 
> tool is needed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper issue #508: ZOOKEEPER-2994 Tool required to recover log and snapsh...

2018-04-25 Thread phunt
Github user phunt commented on the issue:

https://github.com/apache/zookeeper/pull/508
  
+1. Great. Thanks @anmolnar . I've committed for 3.4.13.


---


[jira] [Updated] (ZOOKEEPER-2994) Tool required to recover log and snapshot entries with CRC errors

2018-04-25 Thread Patrick Hunt (JIRA)

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt updated ZOOKEEPER-2994:

Fix Version/s: 3.4.13

> Tool required to recover log and snapshot entries with CRC errors
> -
>
> Key: ZOOKEEPER-2994
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2994
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.5.4, 3.6.0, 3.4.13
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 3.5.4, 3.6.0, 3.4.13
>
>
> In the even that the zookeeper transaction log or snapshot become corrupted 
> and fail CRC checks (preventing startup) we should have a mechanism to get 
> the cluster running again.
> Previously we achieved this by loading the broken transaction log with a 
> modified version of ZK with disabled CRC check and forced it to snapshot.
> It'd very handy to have a tool which can do this for us. LogFormatter and 
> SnapshotFormatter have already been designed to dump log and snapshot files, 
> it'd be nice to extend their functionality and add ability for such recovery.
> It has proven that once you end up with the corrupt txn log there is no way 
> to recover except manually modifying the crc check. That's basically why the 
> tool is needed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ZOOKEEPER-2994) Tool required to recover log and snapshot entries with CRC errors

2018-04-25 Thread Patrick Hunt (JIRA)

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt updated ZOOKEEPER-2994:

Affects Version/s: 3.4.13

> Tool required to recover log and snapshot entries with CRC errors
> -
>
> Key: ZOOKEEPER-2994
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2994
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.5.4, 3.6.0, 3.4.13
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 3.5.4, 3.6.0, 3.4.13
>
>
> In the even that the zookeeper transaction log or snapshot become corrupted 
> and fail CRC checks (preventing startup) we should have a mechanism to get 
> the cluster running again.
> Previously we achieved this by loading the broken transaction log with a 
> modified version of ZK with disabled CRC check and forced it to snapshot.
> It'd very handy to have a tool which can do this for us. LogFormatter and 
> SnapshotFormatter have already been designed to dump log and snapshot files, 
> it'd be nice to extend their functionality and add ability for such recovery.
> It has proven that once you end up with the corrupt txn log there is no way 
> to recover except manually modifying the crc check. That's basically why the 
> tool is needed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ZOOKEEPER-3019) Add a metric to track number of slow fsyncs

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453113#comment-16453113
 ] 

ASF GitHub Bot commented on ZOOKEEPER-3019:
---

Github user phunt commented on the issue:

https://github.com/apache/zookeeper/pull/501
  
I'm afraid this is no longer auto-merging, also the PR build failed on 
jenkins with similar:

> Automatic merge failed; fix conflicts and then commit the result.

@nkalmar could you rebase this on latest master? Thanks.


> Add a metric to track number of slow fsyncs
> ---
>
> Key: ZOOKEEPER-3019
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3019
> Project: ZooKeeper
>  Issue Type: Improvement
>  Components: jmx, server
>Affects Versions: 3.5.3, 3.4.11, 3.6.0
>Reporter: Norbert Kalmar
>Assignee: Norbert Kalmar
>Priority: Major
> Fix For: 3.5.4, 3.6.0, 3.4.13
>
>
> Add jmx bean and Command to ZooKeeper server to expose the the number of slow 
> fsyncs as a metric.
> FileTxnLog.commit() should count the number of times fsync exceeds 
> fsyncWarningThresholdMS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper issue #501: ZOOKEEPER-3019 add metric for slow fsyncs count

2018-04-25 Thread phunt
Github user phunt commented on the issue:

https://github.com/apache/zookeeper/pull/501
  
I'm afraid this is no longer auto-merging, also the PR build failed on 
jenkins with similar:

> Automatic merge failed; fix conflicts and then commit the result.

@nkalmar could you rebase this on latest master? Thanks.


---


[jira] [Commented] (ZOOKEEPER-3007) Potential NPE in ReferenceCountedACLCache#deserialize

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453098#comment-16453098
 ] 

ASF GitHub Bot commented on ZOOKEEPER-3007:
---

Github user phunt commented on the issue:

https://github.com/apache/zookeeper/pull/495
  
Ok, that (call site analysis) makes sense. 

I'm afraid I was unclear, when I said

"You don't need an ERROR in the text here or on the next line."

What I mean is that the text string should not start with "ERROR" given the 
error string is in an exception and the logging (from one of the callers) will 
determine the severity to assign. As such my recommendation would be something 
like:

> throw new RuntimeException("Incorrect format of InputArchive when 
deserialize DataTree - missing acls");

Notice: 1) the removal of "ERROR" and the addition of "missing acls" in 
order to give the person diagnosing the problem a bit more insight (otw they 
have to find the source line in order to get more insight into what they 
formatting issue might be).

If you clear this up (this one line) I think we should be good for commit.

Thanks!


> Potential NPE in ReferenceCountedACLCache#deserialize 
> --
>
> Key: ZOOKEEPER-3007
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3007
> Project: ZooKeeper
>  Issue Type: Bug
>Affects Versions: 3.6.0
>Reporter: lujie
>Priority: Major
>
> Inspired by ZK-3006 , I develop a simple static analysis tool to find other 
> Potential NPE like ZK-3006. This bug is found by this tool ,and I have 
> carefully studied it.  But i am a newbie at here so i may be wrong, hope 
> someone could confirm it and help me improve this tool.
> h3. Bug describtion:
> callee BinaryInputArchive#startVector will return null:
> {code:java}
> // code placeholder
> public Index startVector(String tag) throws IOException {
> int len = readInt(tag);
>  if (len == -1) {
>  return null;
> }
> {code}
> and caller ReferenceCountedACLCache#deserialize  call it without null check
> {code:java}
> // code placeholder
> Index j = ia.startVector("acls");
> while (!j.done()) {
>   ACL acl = new ACL();
>   acl.deserialize(ia, "acl");
> }{code}
> but all the other 14 caller of BinaryInputArchive#startVector performs null 
> checker like:
> {code:java}
> // code placeholder
> Index vidx1 = a_.startVector("acl");
>   if (vidx1!= null)
>  for (; !vidx1.done(); vidx1.incr()){
>  .
> }
>}
> }
> {code}
> so i think we also need add null check in caller 
> ReferenceCountedACLCache#deserialize  just like other 14 caller
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper issue #495: ZOOKEEPER-3007:Potential NPE in ReferenceCountedACLCac...

2018-04-25 Thread phunt
Github user phunt commented on the issue:

https://github.com/apache/zookeeper/pull/495
  
Ok, that (call site analysis) makes sense. 

I'm afraid I was unclear, when I said

"You don't need an ERROR in the text here or on the next line."

What I mean is that the text string should not start with "ERROR" given the 
error string is in an exception and the logging (from one of the callers) will 
determine the severity to assign. As such my recommendation would be something 
like:

> throw new RuntimeException("Incorrect format of InputArchive when 
deserialize DataTree - missing acls");

Notice: 1) the removal of "ERROR" and the addition of "missing acls" in 
order to give the person diagnosing the problem a bit more insight (otw they 
have to find the source line in order to get more insight into what they 
formatting issue might be).

If you clear this up (this one line) I think we should be good for commit.

Thanks!


---


[jira] [Commented] (ZOOKEEPER-2983) Print the classpath when running compile and test ant targets

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453090#comment-16453090
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2983:
---

Github user mfenes closed the pull request at:

https://github.com/apache/zookeeper/pull/471


> Print the classpath when running compile and test ant targets
> -
>
> Key: ZOOKEEPER-2983
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2983
> Project: ZooKeeper
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.5.3, 3.4.11
>Reporter: Mark Fenes
>Assignee: Mark Fenes
>Priority: Major
>
> Printing the classpath helps to verify that we have only the intended 
> classes, jars on the classpath, e.g. clover.jar is included only when running 
> coverage tests.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ZOOKEEPER-2983) Print the classpath when running compile and test ant targets

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453087#comment-16453087
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2983:
---

Github user mfenes closed the pull request at:

https://github.com/apache/zookeeper/pull/470


> Print the classpath when running compile and test ant targets
> -
>
> Key: ZOOKEEPER-2983
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2983
> Project: ZooKeeper
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.5.3, 3.4.11
>Reporter: Mark Fenes
>Assignee: Mark Fenes
>Priority: Major
>
> Printing the classpath helps to verify that we have only the intended 
> classes, jars on the classpath, e.g. clover.jar is included only when running 
> coverage tests.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper pull request #471: ZOOKEEPER-2983: Print the classpath when runnin...

2018-04-25 Thread mfenes
Github user mfenes closed the pull request at:

https://github.com/apache/zookeeper/pull/471


---


[GitHub] zookeeper pull request #470: ZOOKEEPER-2983: Print the classpath when runnin...

2018-04-25 Thread mfenes
Github user mfenes closed the pull request at:

https://github.com/apache/zookeeper/pull/470


---


[jira] [Commented] (ZOOKEEPER-2983) Print the classpath when running compile and test ant targets

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453086#comment-16453086
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2983:
---

Github user mfenes closed the pull request at:

https://github.com/apache/zookeeper/pull/469


> Print the classpath when running compile and test ant targets
> -
>
> Key: ZOOKEEPER-2983
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2983
> Project: ZooKeeper
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.5.3, 3.4.11
>Reporter: Mark Fenes
>Assignee: Mark Fenes
>Priority: Major
>
> Printing the classpath helps to verify that we have only the intended 
> classes, jars on the classpath, e.g. clover.jar is included only when running 
> coverage tests.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper pull request #469: ZOOKEEPER-2983: Print the classpath when runnin...

2018-04-25 Thread mfenes
Github user mfenes closed the pull request at:

https://github.com/apache/zookeeper/pull/469


---


[jira] [Commented] (ZOOKEEPER-1752) Download area contains obsolete releases

2018-04-25 Thread Dave (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453074#comment-16453074
 ] 

Dave commented on ZOOKEEPER-1752:
-

Gotcha. No worries. Was a little confusing during the transition period. We've 
grabbed 3.4.12 instead. Thanks for your help.

> Download area contains obsolete releases
> 
>
> Key: ZOOKEEPER-1752
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1752
> Project: ZooKeeper
>  Issue Type: Bug
>Reporter: Sebb
>Assignee: Patrick Hunt
>Priority: Major
>
> The download area under http://www.apache.org/dist/zookeeper/ contains 
> several superseded releases.
> It is important that only the latest release of each currently maintained 
> product line is stored on the main ASF mirrors. Links to older releases can 
> be provided on download pages, but the links should be to the ASF archive 
> server http://archive.apache.org/dist/zookeeper/
> See http://www.apache.org/dev/release.html#when-to-archive



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Fw: Re: [osgi-dev] Invalid version in bundle org.apache.hadoop.zookeeper=: 3.5.3-beta

2018-04-25 Thread Patrick Hunt
Hi Kevin. Is it the same issue as this?
https://issues.apache.org/jira/browse/ZOOKEEPER-2317

If so perhaps you could take on that JIRA and help drive a solution. I
think the biggest issue is that no-one on our end has sufficient experience
on OSGI to verify a fix.

Regards,

Patrick

On Wed, Apr 25, 2018 at 11:02 AM, Kevin Boyle  wrote:

>  Hello,I'm attempting to use the curator-recipes version 4.0.1 in an OSGi
> application.   The curator version is dependent on zookeeper 3.5.3-beta but
> the version format isn't compatible with OSGI, see the reply from the osgi
> developers forum below, is it possible to replace the "-beta" with ".beta"
> going forward?
> The problem with using an older version is that the last few years include
> a "-alpha" or "-beta" in the dependent zookeeper version.
> Any suggestions on how to avoid this would be appreciated.ThanksK
>
>
>
>
>   
>
>   org.apache.curator
>
>   curator-recipes
>
>   4.0.1
>
>  
>
>
>
>
>
>- Forwarded Message - From: Tim Ward To:
> Kevin Boyle ; OSGi Developer Mail List <
> osgi-...@mail.osgi.org>Sent: Wednesday, April 25, 2018, 1:42:49 PM
> EDTSubject: Re: [osgi-dev] Invalid version in bundle
> org.apache.hadoop.zookeeper=: 3.5.3-beta
>  Hi Kevin,
> OSGi version syntax is strictly defined with four sections separated by
> dots. The first three sections are integers greater than or equal to zero,
> and the final section (the qualifier) is a String.
> In this case the org.apache.hadoop.zookeeper “bundle” has a version of
> 3.5.3-beta, which is not OSGi compatible (3.5.3.beta would be). This is
> because the actual artifact you’re using is malformed - the developers have
> added a Bundle-Version to the artifact which isn’t a valid version. There
> is no way that this has ever been run or tested in OSGi. This is a bug in
> their code and should be raised as such.
> You should be fine using an older version with a legal version number, for
> example 3.4.9.
> I hope this helps,
> Tim
>
> On 25 Apr 2018, at 18:29, Kevin Boyle via osgi-dev 
> wrote:
> I'm new to OSGI and after following the enroute quick start tutorial I
> added one class and one library to the pom.xml included here.   The
> curator-recipes indirectly depends on zookeeper 3.5.3-beta (
> curator-recipes depends on curator-test which depends on zookeeper 3.5.3
> ).  Please help me resolve the version conflict and understand why
> zookeeper 3.5.3-beta is a problem for the quickstart.
> Please help as I'm stuck
> ThanksK
>
>
> package org.osgi.enroute.examples.quickstart.rest;
> import org.apache.curator.framework.CuratorFramework;import
> org.apache.curator.framework.CuratorFrameworkFactory;import
> org.apache.curator.retry.ExponentialBackoffRetry;
> public class StartCurator {  public static void main ( String [] args )  {
> StartCurator s = new StartCurator(); s.start_curator(); }
>  public void start_curator () {  String zookeeper_connection_string =
> "localhost:2181";CuratorFrameworkclient =
> CuratorFrameworkFactory.newClient(zookeeper_connection_string, new
> ExponentialBackoffRetry(1000, 3));System.out.println ("Opening
> curator connection to zookeeper");client.start();
> client.close();System.out.println ("Closing curator connection to
> zookeeper"); }
> }
> 
>
>   org.apache.curator
>
>   curator-recipes
>
>   4.0.1
>
>   
>
> mvn install fails with the following error
> [INFO] 
> 
>
> [ERROR] Failed to execute goal biz.aQute.bnd:bnd-indexer-
> maven-plugin:4.0.0-SNAPSHOT:index (index) on project app: Invalid version
> in bundle org.apache.hadoop.zookeeper=: 3.5.3-beta -> [Help 1]
>
> [ERROR]
>
> [ERROR] To see the full stack trace of the errors, re-run Maven with the
> -e switch.
>
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>
> [ERROR]
>
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
>
> [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/
> MojoExecutionException
>
> [ERROR]
>
> [ERROR] After correcting the problems, you can resume the build with the
> command
>
> [ERROR]   mvn  -rf :app
>
> Kevins-MBP:quickstart kevintboyle$
>
>
>  ___
> OSGi Developer Mail List
> osgi-...@mail.osgi.org
> https://mail.osgi.org/mailman/listinfo/osgi-dev
>
>


[jira] [Commented] (ZOOKEEPER-1752) Download area contains obsolete releases

2018-04-25 Thread Patrick Hunt (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453027#comment-16453027
 ] 

Patrick Hunt commented on ZOOKEEPER-1752:
-

The release is being finalized currently and likely all the steps have not yet 
been completed. [~abrahamfine] is RM for 3.4.12 and driving that. I just jumped 
in to cleanup the downloads as I wasn't sure Abe knew about this Jira or this 
aspect of Apache requirements.

> Download area contains obsolete releases
> 
>
> Key: ZOOKEEPER-1752
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1752
> Project: ZooKeeper
>  Issue Type: Bug
>Reporter: Sebb
>Assignee: Patrick Hunt
>Priority: Major
>
> The download area under http://www.apache.org/dist/zookeeper/ contains 
> several superseded releases.
> It is important that only the latest release of each currently maintained 
> product line is stored on the main ASF mirrors. Links to older releases can 
> be provided on download pages, but the links should be to the ASF archive 
> server http://archive.apache.org/dist/zookeeper/
> See http://www.apache.org/dev/release.html#when-to-archive



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ZOOKEEPER-1752) Download area contains obsolete releases

2018-04-25 Thread Dave (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453011#comment-16453011
 ] 

Dave commented on ZOOKEEPER-1752:
-

Ok. It's still linked to as the single 3.4.x release on the main page under 
documentation.

> Download area contains obsolete releases
> 
>
> Key: ZOOKEEPER-1752
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1752
> Project: ZooKeeper
>  Issue Type: Bug
>Reporter: Sebb
>Assignee: Patrick Hunt
>Priority: Major
>
> The download area under http://www.apache.org/dist/zookeeper/ contains 
> several superseded releases.
> It is important that only the latest release of each currently maintained 
> product line is stored on the main ASF mirrors. Links to older releases can 
> be provided on download pages, but the links should be to the ASF archive 
> server http://archive.apache.org/dist/zookeeper/
> See http://www.apache.org/dev/release.html#when-to-archive



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ZOOKEEPER-1752) Download area contains obsolete releases

2018-04-25 Thread Patrick Hunt (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453010#comment-16453010
 ] 

Patrick Hunt commented on ZOOKEEPER-1752:
-

3.4.11 has a very serious regression which is fixed in 3.4.12. Also 3.4.10 is 
currently the "current" and "stable" link targets.

> Download area contains obsolete releases
> 
>
> Key: ZOOKEEPER-1752
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1752
> Project: ZooKeeper
>  Issue Type: Bug
>Reporter: Sebb
>Assignee: Patrick Hunt
>Priority: Major
>
> The download area under http://www.apache.org/dist/zookeeper/ contains 
> several superseded releases.
> It is important that only the latest release of each currently maintained 
> product line is stored on the main ASF mirrors. Links to older releases can 
> be provided on download pages, but the links should be to the ASF archive 
> server http://archive.apache.org/dist/zookeeper/
> See http://www.apache.org/dev/release.html#when-to-archive



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ZOOKEEPER-1752) Download area contains obsolete releases

2018-04-25 Thread Dave (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453002#comment-16453002
 ] 

Dave commented on ZOOKEEPER-1752:
-

It looks like zookeeper-3.4.11 was removed instead of 3.4.10?

> Download area contains obsolete releases
> 
>
> Key: ZOOKEEPER-1752
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1752
> Project: ZooKeeper
>  Issue Type: Bug
>Reporter: Sebb
>Assignee: Patrick Hunt
>Priority: Major
>
> The download area under http://www.apache.org/dist/zookeeper/ contains 
> several superseded releases.
> It is important that only the latest release of each currently maintained 
> product line is stored on the main ASF mirrors. Links to older releases can 
> be provided on download pages, but the links should be to the ASF archive 
> server http://archive.apache.org/dist/zookeeper/
> See http://www.apache.org/dev/release.html#when-to-archive



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Fw: Re: [osgi-dev] Invalid version in bundle org.apache.hadoop.zookeeper=: 3.5.3-beta

2018-04-25 Thread Kevin Boyle
 Hello,I'm attempting to use the curator-recipes version 4.0.1 in an OSGi 
application.   The curator version is dependent on zookeeper 3.5.3-beta but the 
version format isn't compatible with OSGI, see the reply from the osgi 
developers forum below, is it possible to replace the "-beta" with ".beta" 
going forward?
The problem with using an older version is that the last few years include a 
"-alpha" or "-beta" in the dependent zookeeper version.
Any suggestions on how to avoid this would be appreciated.ThanksK


 

  

      org.apache.curator

      curator-recipes

      4.0.1

 





   - Forwarded Message - From: Tim Ward To: Kevin 
Boyle ; OSGi Developer Mail List 
Sent: Wednesday, April 25, 2018, 1:42:49 PM EDTSubject: 
Re: [osgi-dev] Invalid version in bundle org.apache.hadoop.zookeeper=: 
3.5.3-beta
 Hi Kevin,
OSGi version syntax is strictly defined with four sections separated by dots. 
The first three sections are integers greater than or equal to zero, and the 
final section (the qualifier) is a String.
In this case the org.apache.hadoop.zookeeper “bundle” has a version of 
3.5.3-beta, which is not OSGi compatible (3.5.3.beta would be). This is because 
the actual artifact you’re using is malformed - the developers have added a 
Bundle-Version to the artifact which isn’t a valid version. There is no way 
that this has ever been run or tested in OSGi. This is a bug in their code and 
should be raised as such.
You should be fine using an older version with a legal version number, for 
example 3.4.9.
I hope this helps,
Tim

On 25 Apr 2018, at 18:29, Kevin Boyle via osgi-dev  
wrote:
I'm new to OSGI and after following the enroute quick start tutorial I added 
one class and one library to the pom.xml included here.   The curator-recipes 
indirectly depends on zookeeper 3.5.3-beta ( curator-recipes depends on 
curator-test which depends on zookeeper 3.5.3 ).  Please help me resolve the 
version conflict and understand why zookeeper 3.5.3-beta is a problem for the 
quickstart. 
Please help as I'm stuck
ThanksK


package org.osgi.enroute.examples.quickstart.rest;
import org.apache.curator.framework.CuratorFramework;import 
org.apache.curator.framework.CuratorFrameworkFactory;import 
org.apache.curator.retry.ExponentialBackoffRetry;
public class StartCurator {  public static void main ( String [] args )  { 
StartCurator s = new StartCurator(); s.start_curator(); }
 public void start_curator () {      String zookeeper_connection_string = 
"localhost:2181";        CuratorFramework    client = 
CuratorFrameworkFactory.newClient(zookeeper_connection_string, new 
ExponentialBackoffRetry(1000, 3));        System.out.println ("Opening curator 
connection to zookeeper");        client.start();        client.close();        
System.out.println ("Closing curator connection to zookeeper"); }
}


      org.apache.curator

      curator-recipes

      4.0.1

  

mvn install fails with the following error
[INFO] 

[ERROR] Failed to execute goal 
biz.aQute.bnd:bnd-indexer-maven-plugin:4.0.0-SNAPSHOT:index (index) on project 
app: Invalid version in bundle org.apache.hadoop.zookeeper=: 3.5.3-beta -> 
[Help 1]

[ERROR] 

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please 
read the following articles:

[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

[ERROR] 

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn  -rf :app

Kevins-MBP:quickstart kevintboyle$ 
    

 ___
OSGi Developer Mail List
osgi-...@mail.osgi.org
https://mail.osgi.org/mailman/listinfo/osgi-dev

  

Re: Please add your release data for 'zookeeper'

2018-04-25 Thread Patrick Hunt
I took care of this. Thanks Abe!

Patrick

On Tue, Apr 24, 2018 at 8:40 PM, Abraham Fine  wrote:

> Hello PMC Members-
>
> Would one of you please help me with this part of the release process?
>
> Thanks,
> Abe
>
> - Original message -
> From: Apache Reporter Service 
> To: afine 
> Subject: Please add your release data for 'zookeeper'
> Date: Wed, 25 Apr 2018 03:30:34 + (UTC)
>
> Hi,
> This is an automated email from reporter.apache.org.
> I see that you just pushed something to our release repository for the
> 'zookeeper' project
> in the following commit:
>
> r26497 at 2018-04-25 03:21:58 + (Wed, 25 Apr 2018)
> Add ZooKeeper 3.4.12 release
>
> If you are a PMC member of this project, we ask that you log on to:
> https://reporter.apache.org/addrelease.html?zookeeper
> and add your release data (version and date) to the database.
>
> If you are not a PMC member, please have a PMC member add this information.
>
> While this is not a requirement, we ask that you still add this data to the
> reporter database, so that people using the Apache Reporter Service will be
> able to see the latest release data for this project.
>
> Also, please ensure that you remove [1] any older releases.
>
> With regards,
> The Apache Reporter Service.
>
> [1] http://www.apache.org/dev/release.html#when-to-archive
>
>


[jira] [Updated] (ZOOKEEPER-3026) ReadOnlyModeTest is using Thread deprecated API.

2018-04-25 Thread Patrick Hunt (JIRA)

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt updated ZOOKEEPER-3026:

Description: 
Same issue as ZOOKEEPER-2415

 

Suspend and resume are being called on peers (which are subclasses of Thread):
{quote}// if we don't suspend a peer it will rejoin a quorum
 qu.getPeer(1).peer.suspend();



// resume poor fellow
 qu.getPeer(1).peer.resume();
{quote}

  was:Same issue as ZOOKEEPER-2415


> ReadOnlyModeTest  is using Thread deprecated API.
> -
>
> Key: ZOOKEEPER-3026
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3026
> Project: ZooKeeper
>  Issue Type: Bug
>Affects Versions: 3.5.4, 3.6.0, 3.4.12
>Reporter: Patrick Hunt
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 3.5.4, 3.6.0, 3.4.13
>
>
> Same issue as ZOOKEEPER-2415
>  
> Suspend and resume are being called on peers (which are subclasses of Thread):
> {quote}// if we don't suspend a peer it will rejoin a quorum
>  qu.getPeer(1).peer.suspend();
> 
> // resume poor fellow
>  qu.getPeer(1).peer.resume();
> {quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ZOOKEEPER-3026) ReadOnlyModeTest is using Thread deprecated API.

2018-04-25 Thread Patrick Hunt (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452343#comment-16452343
 ] 

Patrick Hunt commented on ZOOKEEPER-3026:
-

Ah, sorry about that! I will update the description. My bad.

> ReadOnlyModeTest  is using Thread deprecated API.
> -
>
> Key: ZOOKEEPER-3026
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3026
> Project: ZooKeeper
>  Issue Type: Bug
>Affects Versions: 3.5.4, 3.6.0, 3.4.12
>Reporter: Patrick Hunt
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 3.5.4, 3.6.0, 3.4.13
>
>
> Same issue as ZOOKEEPER-2415



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Success: ZOOKEEPER- PreCommit Build #1630

2018-04-25 Thread Apache Jenkins Server
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1630/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 41.08 MB...]
 [exec] +1 @author.  The patch does not contain any @author tags.
 [exec] 
 [exec] +1 tests included.  The patch appears to include 13 new or 
modified tests.
 [exec] 
 [exec] +1 javadoc.  The javadoc tool did not generate any warning 
messages.
 [exec] 
 [exec] +1 javac.  The applied patch does not increase the total number 
of javac compiler warnings.
 [exec] 
 [exec] +1 findbugs.  The patch does not introduce any new Findbugs 
(version 3.0.1) warnings.
 [exec] 
 [exec] +1 release audit.  The applied patch does not increase the 
total number of release audit warnings.
 [exec] 
 [exec] +1 core tests.  The patch passed core unit tests.
 [exec] 
 [exec] +1 contrib tests.  The patch passed contrib unit tests.
 [exec] 
 [exec] Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1630//testReport/
 [exec] Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1630//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
 [exec] Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1630//console
 [exec] 
 [exec] This message is automatically generated.
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Adding comment to Jira.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] Comment with id 16452229 added to ZOOKEEPER-2994.
 [exec] Session logged out. Session was 
JSESSIONID=D2169C24711B16329F95BF5E1390DA0C.
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Finished build.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] mv: 
'/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess'
 and 
'/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess'
 are the same file

BUILD SUCCESSFUL
Total time: 34 minutes 16 seconds
Archiving artifacts
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8
Recording test results
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8
[description-setter] Description set: ZOOKEEPER-2994
Putting comment on the pull request
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8
Email was triggered for: Success
Sending email for trigger: Success
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8
Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8



###
## FAILED TESTS (if any) 
##
All tests passed

[jira] [Commented] (ZOOKEEPER-2994) Tool required to recover log and snapshot entries with CRC errors

2018-04-25 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452229#comment-16452229
 ] 

Hadoop QA commented on ZOOKEEPER-2994:
--

+1 overall.  GitHub Pull Request  Build
  

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 13 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 3.0.1) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1630//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1630//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1630//console

This message is automatically generated.

> Tool required to recover log and snapshot entries with CRC errors
> -
>
> Key: ZOOKEEPER-2994
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2994
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.5.4, 3.6.0
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 3.5.4, 3.6.0
>
>
> In the even that the zookeeper transaction log or snapshot become corrupted 
> and fail CRC checks (preventing startup) we should have a mechanism to get 
> the cluster running again.
> Previously we achieved this by loading the broken transaction log with a 
> modified version of ZK with disabled CRC check and forced it to snapshot.
> It'd very handy to have a tool which can do this for us. LogFormatter and 
> SnapshotFormatter have already been designed to dump log and snapshot files, 
> it'd be nice to extend their functionality and add ability for such recovery.
> It has proven that once you end up with the corrupt txn log there is no way 
> to recover except manually modifying the crc check. That's basically why the 
> tool is needed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Failed: ZOOKEEPER- PreCommit Build #1631

2018-04-25 Thread Apache Jenkins Server
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1631/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 2.82 KB...]
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at 
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:281)
at com.sun.proxy.$Proxy110.merge(Unknown Source)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl.merge(RemoteGitImpl.java:435)
at 
com.cloudbees.jenkins.plugins.git.vmerge.BuildChooserImpl.getCandidateRevisions(BuildChooserImpl.java:107)
at 
hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:1054)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1161)
at hudson.scm.SCM.checkout(SCM.java:495)
at 
hudson.model.AbstractProject.checkout(AbstractProject.java:1202)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at 
jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1724)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at 
hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
hudson.plugins.git.GitException: Command "git merge 
089bb754f4199cc58d9df48c75b82b461c54e42f" returned status code 1:
stdout: CONFLICT (rename/delete): 
zk-server/test/org/apache/zookeeper/ZooKeeperTestableTest.java deleted in HEAD 
and renamed in 089bb754f4199cc58d9df48c75b82b461c54e42f. Version 
089bb754f4199cc58d9df48c75b82b461c54e42f of 
zk-server/test/org/apache/zookeeper/ZooKeeperTestableTest.java left in tree.
Auto-merging zk-contrib/build.xml
Removing src/java/main/org/apache/jute/compiler/generated/rcc.jj
Removing src/java/main/org/apache/jute/compiler/generated/package.html
Removing src/java/OldChangeLog
Removing src/contrib/zkperl/build/check_zk_version.h
Removing src/contrib/zkperl/build/check_zk_version.c
Removing src/NOTICE.txt
Removing src/LICENSE.txt
Automatic merge failed; fix conflicts and then commit the result.

stderr: 
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1996)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1964)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1960)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1597)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl$3.execute(CliGitAPIImpl.java:697)
at 
org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.merge(AbstractGitAPIImpl.java:86)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.merge(CliGitAPIImpl.java:72)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:922)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:896)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:853)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Archiving artifacts
Recording test results
ERROR: Step ?Publish JUnit test result report? failed: No test report files 
were found. Configuration error?
[description-setter] Could not determine description.
Putting comment on the pull request
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
No tests ran.

[jira] [Commented] (ZOOKEEPER-3026) ReadOnlyModeTest is using Thread deprecated API.

2018-04-25 Thread Andor Molnar (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452171#comment-16452171
 ] 

Andor Molnar commented on ZOOKEEPER-3026:
-

[~phunt] Sure, no problem. Thanks for creating this Jira.

I checked ReadOnlyModeTest, but I cannot see which deprecated Thread api you're 
referring to. I can only see some Thread.sleep()s in the code, but afaik that's 
not deprecated. Would you please elaborate a little bit?

> ReadOnlyModeTest  is using Thread deprecated API.
> -
>
> Key: ZOOKEEPER-3026
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3026
> Project: ZooKeeper
>  Issue Type: Bug
>Affects Versions: 3.5.4, 3.6.0, 3.4.12
>Reporter: Patrick Hunt
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 3.5.4, 3.6.0, 3.4.13
>
>
> Same issue as ZOOKEEPER-2415



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ZOOKEEPER-2994) Tool required to recover log and snapshot entries with CRC errors

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452162#comment-16452162
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2994:
---

Github user anmolnar commented on the issue:

https://github.com/apache/zookeeper/pull/508
  
@phunt Done. Now it's using a custom parser and no additional dependency 
requirement.


> Tool required to recover log and snapshot entries with CRC errors
> -
>
> Key: ZOOKEEPER-2994
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2994
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.5.4, 3.6.0
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 3.5.4, 3.6.0
>
>
> In the even that the zookeeper transaction log or snapshot become corrupted 
> and fail CRC checks (preventing startup) we should have a mechanism to get 
> the cluster running again.
> Previously we achieved this by loading the broken transaction log with a 
> modified version of ZK with disabled CRC check and forced it to snapshot.
> It'd very handy to have a tool which can do this for us. LogFormatter and 
> SnapshotFormatter have already been designed to dump log and snapshot files, 
> it'd be nice to extend their functionality and add ability for such recovery.
> It has proven that once you end up with the corrupt txn log there is no way 
> to recover except manually modifying the crc check. That's basically why the 
> tool is needed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper issue #508: ZOOKEEPER-2994 Tool required to recover log and snapsh...

2018-04-25 Thread anmolnar
Github user anmolnar commented on the issue:

https://github.com/apache/zookeeper/pull/508
  
@phunt Done. Now it's using a custom parser and no additional dependency 
requirement.


---


[jira] [Commented] (ZOOKEEPER-3019) Add a metric to track number of slow fsyncs

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452134#comment-16452134
 ] 

ASF GitHub Bot commented on ZOOKEEPER-3019:
---

Github user nkalmar commented on the issue:

https://github.com/apache/zookeeper/pull/501
  
Removed fsyncExceed from stats command, but the tracking variable still 
remains in ServerStats. I was thinking of moving it to for example zkDb, but 
that wouldn't make any more sense in my opinion. 
The other option is to leave it in FileTxnLog as a static variable, that 
was my original idea actually. 
So if that is a better solution, I will change it.


> Add a metric to track number of slow fsyncs
> ---
>
> Key: ZOOKEEPER-3019
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3019
> Project: ZooKeeper
>  Issue Type: Improvement
>  Components: jmx, server
>Affects Versions: 3.5.3, 3.4.11, 3.6.0
>Reporter: Norbert Kalmar
>Assignee: Norbert Kalmar
>Priority: Major
> Fix For: 3.5.4, 3.6.0, 3.4.13
>
>
> Add jmx bean and Command to ZooKeeper server to expose the the number of slow 
> fsyncs as a metric.
> FileTxnLog.commit() should count the number of times fsync exceeds 
> fsyncWarningThresholdMS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper issue #501: ZOOKEEPER-3019 add metric for slow fsyncs count

2018-04-25 Thread nkalmar
Github user nkalmar commented on the issue:

https://github.com/apache/zookeeper/pull/501
  
Removed fsyncExceed from stats command, but the tracking variable still 
remains in ServerStats. I was thinking of moving it to for example zkDb, but 
that wouldn't make any more sense in my opinion. 
The other option is to leave it in FileTxnLog as a static variable, that 
was my original idea actually. 
So if that is a better solution, I will change it.


---


Failed: ZOOKEEPER- PreCommit Build #1629

2018-04-25 Thread Apache Jenkins Server
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1629/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 2.23 KB...]

stderr: 
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to H0
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at 
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:281)
at com.sun.proxy.$Proxy110.merge(Unknown Source)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl.merge(RemoteGitImpl.java:435)
at 
com.cloudbees.jenkins.plugins.git.vmerge.BuildChooserImpl.getCandidateRevisions(BuildChooserImpl.java:107)
at 
hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:1054)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1161)
at hudson.scm.SCM.checkout(SCM.java:495)
at 
hudson.model.AbstractProject.checkout(AbstractProject.java:1202)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at 
jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1724)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at 
hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
hudson.plugins.git.GitException: Command "git merge 
8f3ea1c0e545df3dc98fbb46acedd7738b439aa0" returned status code 1:
stdout: Auto-merging 
src/java/test/org/apache/zookeeper/server/persistence/FileTxnLogTest.java
Auto-merging 
src/java/main/org/apache/zookeeper/server/persistence/FileTxnLog.java
CONFLICT (content): Merge conflict in 
src/java/main/org/apache/zookeeper/server/persistence/FileTxnLog.java
Auto-merging src/docs/src/documentation/content/xdocs/zookeeperAdmin.xml
Auto-merging build.xml
Automatic merge failed; fix conflicts and then commit the result.

stderr: 
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1996)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1964)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1960)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1597)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl$3.execute(CliGitAPIImpl.java:697)
at 
org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.merge(AbstractGitAPIImpl.java:86)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.merge(CliGitAPIImpl.java:72)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:922)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:896)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:853)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Archiving artifacts
Recording test results
ERROR: Step ?Publish JUnit test result report? failed: No test report files 
were found. Configuration error?
[description-setter] Could not determine description.
Putting comment on the pull request
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
No tests ran.

[jira] [Commented] (ZOOKEEPER-2415) SessionTest is using Thread deprecated API.

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451932#comment-16451932
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2415:
---

Github user anmolnar closed the pull request at:

https://github.com/apache/zookeeper/pull/506


> SessionTest is using Thread deprecated API.
> ---
>
> Key: ZOOKEEPER-2415
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2415
> Project: ZooKeeper
>  Issue Type: Test
>  Components: tests
>Affects Versions: 3.4.8, 3.5.1, 3.6.0
>Reporter: Flavio Junqueira
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 3.5.4, 3.6.0, 3.4.13
>
>
> The test class is using calls such as {{Thread.suspend}} and 
> {{Thread.resume}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ZOOKEEPER-2994) Tool required to recover log and snapshot entries with CRC errors

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451931#comment-16451931
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2994:
---

Github user anmolnar commented on the issue:

https://github.com/apache/zookeeper/pull/508
  
I will rather remove the dependency and change the implementation to behave 
similarly without it, so that we can merge this safely.


> Tool required to recover log and snapshot entries with CRC errors
> -
>
> Key: ZOOKEEPER-2994
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2994
> Project: ZooKeeper
>  Issue Type: New Feature
>Affects Versions: 3.5.4, 3.6.0
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 3.5.4, 3.6.0
>
>
> In the even that the zookeeper transaction log or snapshot become corrupted 
> and fail CRC checks (preventing startup) we should have a mechanism to get 
> the cluster running again.
> Previously we achieved this by loading the broken transaction log with a 
> modified version of ZK with disabled CRC check and forced it to snapshot.
> It'd very handy to have a tool which can do this for us. LogFormatter and 
> SnapshotFormatter have already been designed to dump log and snapshot files, 
> it'd be nice to extend their functionality and add ability for such recovery.
> It has proven that once you end up with the corrupt txn log there is no way 
> to recover except manually modifying the crc check. That's basically why the 
> tool is needed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper pull request #506: ZOOKEEPER-2415 SessionTest is using Thread depr...

2018-04-25 Thread anmolnar
Github user anmolnar closed the pull request at:

https://github.com/apache/zookeeper/pull/506


---


[GitHub] zookeeper issue #508: ZOOKEEPER-2994 Tool required to recover log and snapsh...

2018-04-25 Thread anmolnar
Github user anmolnar commented on the issue:

https://github.com/apache/zookeeper/pull/508
  
I will rather remove the dependency and change the implementation to behave 
similarly without it, so that we can merge this safely.


---


[jira] [Commented] (ZOOKEEPER-3019) Add a metric to track number of slow fsyncs

2018-04-25 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451876#comment-16451876
 ] 

ASF GitHub Bot commented on ZOOKEEPER-3019:
---

Github user nkalmar commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/501#discussion_r183980318
  
--- Diff: src/java/test/org/apache/zookeeper/test/FourLetterWordsTest.java 
---
@@ -167,6 +167,8 @@ public void testValidateStatOutput() throws Exception {
 line = in.readLine();
 Assert.assertTrue(Pattern.matches("^Mode: .*$", line));
 line = in.readLine();
+Assert.assertTrue(Pattern.matches("^Fsync threshold exceeded: 
\\d+$", line));
--- End diff --

Yes, it would have been an ugly solution, that's why I didn't go ahead with 
it.
Okay, sounds good to me, I will make the changes.

Thanks!


> Add a metric to track number of slow fsyncs
> ---
>
> Key: ZOOKEEPER-3019
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3019
> Project: ZooKeeper
>  Issue Type: Improvement
>  Components: jmx, server
>Affects Versions: 3.5.3, 3.4.11, 3.6.0
>Reporter: Norbert Kalmar
>Assignee: Norbert Kalmar
>Priority: Major
> Fix For: 3.5.4, 3.6.0, 3.4.13
>
>
> Add jmx bean and Command to ZooKeeper server to expose the the number of slow 
> fsyncs as a metric.
> FileTxnLog.commit() should count the number of times fsync exceeds 
> fsyncWarningThresholdMS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] zookeeper pull request #501: ZOOKEEPER-3019 add metric for slow fsyncs count

2018-04-25 Thread nkalmar
Github user nkalmar commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/501#discussion_r183980318
  
--- Diff: src/java/test/org/apache/zookeeper/test/FourLetterWordsTest.java 
---
@@ -167,6 +167,8 @@ public void testValidateStatOutput() throws Exception {
 line = in.readLine();
 Assert.assertTrue(Pattern.matches("^Mode: .*$", line));
 line = in.readLine();
+Assert.assertTrue(Pattern.matches("^Fsync threshold exceeded: 
\\d+$", line));
--- End diff --

Yes, it would have been an ugly solution, that's why I didn't go ahead with 
it.
Okay, sounds good to me, I will make the changes.

Thanks!


---