[jira] [Commented] (ZOOKEEPER-2186) QuorumCnxManager#receiveConnection may crash with random input
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537663#comment-14537663 ] Hadoop QA commented on ZOOKEEPER-2186: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12731857/ZOOKEEPER-2186.patch against trunk revision 1678645. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2685//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2685//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2685//console This message is automatically generated. QuorumCnxManager#receiveConnection may crash with random input -- Key: ZOOKEEPER-2186 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2186 Project: ZooKeeper Issue Type: Bug Components: server Affects Versions: 3.4.6, 3.5.0 Reporter: Raul Gutierrez Segales Assignee: Raul Gutierrez Segales Fix For: 3.4.7, 3.5.1, 3.6.0 Attachments: ZOOKEEPER-2186.patch This will allocate an arbitrarily large byte buffer (and try to read it!): {code} public boolean receiveConnection(Socket sock) { Long sid = null; ... sid = din.readLong(); // next comes the #bytes in the remainder of the message int num_remaining_bytes = din.readInt(); byte[] b = new byte[num_remaining_bytes]; // remove the remainder of the message from din int num_read = din.read(b); {code} This will crash the QuorumCnxManager thread, so the cluster will keep going but future elections might fail to converge (ditto for leaving/joining members). Patch coming up in a bit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Success: ZOOKEEPER-2186 PreCommit Build #2685
Jira: https://issues.apache.org/jira/browse/ZOOKEEPER-2186 Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2685/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 370583 lines...] [exec] +1 overall. Here are the results of testing the latest attachment [exec] http://issues.apache.org/jira/secure/attachment/12731857/ZOOKEEPER-2186.patch [exec] against trunk revision 1678645. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 core tests. The patch passed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2685//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2685//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2685//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] 79cafc99528280b11e0960d34e391e1d7a156ff3 logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] BUILD SUCCESSFUL Total time: 48 minutes 13 seconds Archiving artifacts Sending artifact delta relative to PreCommit-ZOOKEEPER-Build #2683 Archived 24 artifacts Archive block size is 32768 Received 3 blocks and 33661975 bytes Compression is 0.3% Took 12 sec Recording test results Description set: ZOOKEEPER-2186 Email was triggered for: Success Sending email for trigger: Success ### ## FAILED TESTS (if any) ## All tests passed
ZooKeeper-trunk-WinVS2008 - Build # 1798 - Still Failing
See https://builds.apache.org/job/ZooKeeper-trunk-WinVS2008/1798/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 112 lines...] [javacc] File SimpleCharStream.java does not exist. Will create one. [javacc] Parser generated successfully. jute: [javac] Compiling 39 source files to f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008\trunk\build\classes compile_jute_uptodate: compile_jute: [mkdir] Created dir: f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008\trunk\src\java\generated [mkdir] Created dir: f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008\trunk\src\c\generated [java] ../../zookeeper.jute Parsed Successfully [java] ../../zookeeper.jute Parsed Successfully [touch] Creating f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008\trunk\src\java\generated\.generated BUILD SUCCESSFUL Total time: 9 seconds [ZooKeeper-trunk-WinVS2008] $ cmd /c call C:\Windows\TEMP\hudson2378915579418469026.bat f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008set ZOOKEEPER_HOME=f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008\trunk f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008msbuild trunk/src/c/zookeeper.sln /p:Configuration=Release Microsoft (R) Build Engine version 4.0.30319.18408 [Microsoft .NET Framework, version 4.0.30319.18444] Copyright (C) Microsoft Corporation. All rights reserved. Building the projects in this solution one at a time. To enable parallel build, please add the /m switch. Build started 5/11/2015 9:22:25 AM. Project f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008\trunk\src\c\zookeeper.sln on node 1 (default targets). ValidateSolutionConfiguration: Building solution configuration Release|Win32. zookeeper: C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\..\..\vc\vcpackages\VCBuild.exe f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008\trunk\src\c\zookeeper.vcproj Release|Win32 cl : Command line warning D9035: option 'Wp64' has been deprecated and will be removed in a future release [f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008\trunk\src\c\zookeeper.sln] .\src\zookeeper.c(43): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory [f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008\trunk\src\c\zookeeper.sln] Cli: C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\..\..\vc\vcpackages\VCBuild.exe f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008\trunk\src\c\Cli.vcproj Release|Win32 Done Building Project f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008\trunk\src\c\zookeeper.sln (default targets) -- FAILED. Build FAILED. f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008\trunk\src\c\zookeeper.sln (default target) (1) - (zookeeper target) - cl : Command line warning D9035: option 'Wp64' has been deprecated and will be removed in a future release [f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008\trunk\src\c\zookeeper.sln] f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008\trunk\src\c\zookeeper.sln (default target) (1) - (zookeeper target) - .\src\zookeeper.c(43): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory [f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008\trunk\src\c\zookeeper.sln] 1 Warning(s) 1 Error(s) Time Elapsed 00:00:05.73 f:\hudson\hudson-slave\workspace\ZooKeeper-trunk-WinVS2008exit 1 Build step 'Execute Windows batch command' marked build as failure Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
ZooKeeper-trunk-solaris - Build # 1028 - Still Failing
See https://builds.apache.org/job/ZooKeeper-trunk-solaris/1028/ ### ## LAST 60 LINES OF THE CONSOLE ### Started by timer Building remotely on solaris1 (Solaris) in workspace /export/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris Updating http://svn.apache.org/repos/asf/zookeeper/trunk at revision '2015-05-11T08:31:08.558 +' D src/java/test/org/apache/zookeeper/server/quorum/ReconfigFailureCases.java A src/java/test/org/apache/zookeeper/server/quorum/ReconfigFailureCasesTest.java D src/java/test/org/apache/zookeeper/test/ClientRetry.java D src/java/test/org/apache/zookeeper/test/WatchEventWhenAutoReset.java A src/java/test/org/apache/zookeeper/test/ClientRetryTest.java A src/java/test/org/apache/zookeeper/test/WatchEventWhenAutoResetTest.java U CHANGES.txt At revision 1678689 Updating http://svn.apache.org/repos/asf/hadoop/nightly at revision '2015-05-11T08:31:08.558 +' At revision 1678689 no change for http://svn.apache.org/repos/asf/hadoop/nightly since the previous build No emails were triggered. [locks-and-latches] Checking to see if we really have the locks [locks-and-latches] Have all the locks, build can start [ZooKeeper-trunk-solaris] $ /bin/bash /var/tmp/hudson2574898498699224501.sh /var/tmp/hudson2574898498699224501.sh: line 12: ant: command not found Build step 'Execute shell' marked build as failure [locks-and-latches] Releasing all the locks [locks-and-latches] All the locks released Recording test results Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
[jira] [Commented] (ZOOKEEPER-2188) client connection hung up because of dead loop
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537821#comment-14537821 ] sunhaitao commented on ZOOKEEPER-2188: -- shuting down the cluster is just one case, another case is like in safe authentication failed, the clientCnxn just keeps trying to connect but no feed back. client connection hung up because of dead loop --- Key: ZOOKEEPER-2188 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2188 Project: ZooKeeper Issue Type: Bug Components: java client Affects Versions: 3.5.0 Reporter: sunhaitao There is something wrong with the client code ClientCnxn.java, it will keep trying to connect to server in a dead loop. This is my test step, shut down zookeeper cluster, exectue zkCli.sh script to connect to zookeeper cluster, it will keep trying to connect to zookeeper server without stop. public void run() { clientCnxnSocket.introduce(this, sessionId, outgoingQueue); clientCnxnSocket.updateNow(); clientCnxnSocket.updateLastSendAndHeard(); int to; long lastPingRwServer = Time.currentElapsedTime(); final int MAX_SEND_PING_INTERVAL = 1; //10 seconds while (state.isAlive()) { try { if (!clientCnxnSocket.isConnected()) { // don't re-establish connection if we are closing if (closing) { break; } startConnect(); clientCnxnSocket.updateLastSendAndHeard(); } public boolean isAlive() { return this != CLOSED this != AUTH_FAILED; } because at the beginning it is CONNECTING so isAlive always returns true, which leads to dead loop. we should add some retry limit to stop this -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-2182) Several test suites are not running during pre-commit, because their names do not end with Test.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537827#comment-14537827 ] Hudson commented on ZOOKEEPER-2182: --- SUCCESS: Integrated in ZooKeeper-trunk #2688 (See [https://builds.apache.org/job/ZooKeeper-trunk/2688/]) ZOOKEEPER-2182 Several test suites are not running during pre-commit, because their names do not end with Test. (Chris Nauroth via hdeng) (hdeng: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1678645) * /zookeeper/trunk/CHANGES.txt * /zookeeper/trunk/src/java/test/org/apache/zookeeper/server/quorum/ReconfigFailureCases.java * /zookeeper/trunk/src/java/test/org/apache/zookeeper/server/quorum/ReconfigFailureCasesTest.java * /zookeeper/trunk/src/java/test/org/apache/zookeeper/test/ClientRetry.java * /zookeeper/trunk/src/java/test/org/apache/zookeeper/test/ClientRetryTest.java * /zookeeper/trunk/src/java/test/org/apache/zookeeper/test/WatchEventWhenAutoReset.java * /zookeeper/trunk/src/java/test/org/apache/zookeeper/test/WatchEventWhenAutoResetTest.java Several test suites are not running during pre-commit, because their names do not end with Test. -- Key: ZOOKEEPER-2182 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2182 Project: ZooKeeper Issue Type: Bug Components: tests Affects Versions: 3.5.0 Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 3.5.1, 3.6.0 Attachments: ZOOKEEPER-2182.001.patch In build.xml, the {{junit}} task definition uses an include pattern of {{\*\*/\*$\{test.category\}Test.java}}. This is important so that we don't accidentally try to run utility classes like {{PortAssignment}} or {{TestableZooKeeper}} as if they were JUnit suites. However, several test suites are misnamed so that they don't satisfy this pattern, and therefore pre-commit hasn't been running them. {{ClientRetry}} {{ReconfigFailureCases}} {{WatchEventWhenAutoReset}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-2186) QuorumCnxManager#receiveConnection may crash with random input
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538301#comment-14538301 ] Raul Gutierrez Segales commented on ZOOKEEPER-2186: --- [~hdeng]: git-review is broken for me today, mind reviewing in the PR: https://github.com/apache/zookeeper/pull/30 ? Thanks! QuorumCnxManager#receiveConnection may crash with random input -- Key: ZOOKEEPER-2186 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2186 Project: ZooKeeper Issue Type: Bug Components: server Affects Versions: 3.4.6, 3.5.0 Reporter: Raul Gutierrez Segales Assignee: Raul Gutierrez Segales Fix For: 3.4.7, 3.5.1, 3.6.0 Attachments: ZOOKEEPER-2186.patch This will allocate an arbitrarily large byte buffer (and try to read it!): {code} public boolean receiveConnection(Socket sock) { Long sid = null; ... sid = din.readLong(); // next comes the #bytes in the remainder of the message int num_remaining_bytes = din.readInt(); byte[] b = new byte[num_remaining_bytes]; // remove the remainder of the message from din int num_read = din.read(b); {code} This will crash the QuorumCnxManager thread, so the cluster will keep going but future elections might fail to converge (ditto for leaving/joining members). Patch coming up in a bit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-2186) QuorumCnxManager#receiveConnection may crash with random input
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538609#comment-14538609 ] Alexander Shraer commented on ZOOKEEPER-2186: - Thanks Raul! I was indeed not very careful here. The patch looks good to me overall. Consider reducing the max buffer size - b only contains a string of the form host:port, so jute.maxbuffer and 1 MB may be way too much. QuorumCnxManager#receiveConnection may crash with random input -- Key: ZOOKEEPER-2186 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2186 Project: ZooKeeper Issue Type: Bug Components: server Affects Versions: 3.4.6, 3.5.0 Reporter: Raul Gutierrez Segales Assignee: Raul Gutierrez Segales Fix For: 3.4.7, 3.5.1, 3.6.0 Attachments: ZOOKEEPER-2186.patch This will allocate an arbitrarily large byte buffer (and try to read it!): {code} public boolean receiveConnection(Socket sock) { Long sid = null; ... sid = din.readLong(); // next comes the #bytes in the remainder of the message int num_remaining_bytes = din.readInt(); byte[] b = new byte[num_remaining_bytes]; // remove the remainder of the message from din int num_read = din.read(b); {code} This will crash the QuorumCnxManager thread, so the cluster will keep going but future elections might fail to converge (ditto for leaving/joining members). Patch coming up in a bit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ZOOKEEPER-2183) Change test port assignments to improve uniqueness of ports for multiple concurrent test processes on the same host.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated ZOOKEEPER-2183: - Attachment: ZOOKEEPER-2183.001.patch I'm attaching a patch. Here is a summary of the changes: # build.xml: Add support for concurrent execution of tests. The default is still a single JUnit process (same as today), but you can override it by passing a command line argument. This would require everyone to upgrade to Ant 1.9.4. I don't know what version Jenkins runs, so if it fails on this, then I'll provide another patch without the build.xml change. On my fairly wimpy CentOS VM, I can do a full test run with {{ant -Dtest.junit.threads=8 -Dtest.output=yes clean test-core-java}} in ~10-15 minutes, down from ~60 minutes. # {{PortAssignment}}: I rewrote this to assign ports by binding to an ephemeral port temporarily. I documented that there is still a very small race condition risk, but it's going to be much better than the current situation where multiple JUnit processes starting from the same base port number could collide. I did explore trying to bootstrap test servers with port 0 to eliminate this race condition completely, but that would have required intrusive changes in product code and tests that expect to know all their port numbers upfront before running anything. # Multiple test suites are updated to skip starting {{JettyAdminServer}}, so that we don't get a {{BindException}} when they all try to use the same port. All test suites that inherit from {{ZKTestCase}} already do this. # Multiple tests are updated to remove + 1000 logic from their port assignments. This looks like it was an attempt to offset port numbers a bit in certain tests to try to increase the probability of uniqueness. This is no longer necessary with the new {{PortAssignment}}. Also, the logic is incorrect, because it could inadvertently cause a port collision or overflow past the valid range for port numbers. I've tried multiple concurrent test runs with this patch, and there are no signs of a {{BindException}}, so I think this covers the changes we need in port assignments. Change test port assignments to improve uniqueness of ports for multiple concurrent test processes on the same host. Key: ZOOKEEPER-2183 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2183 Project: ZooKeeper Issue Type: Improvement Components: tests Reporter: Chris Nauroth Assignee: Chris Nauroth Attachments: ZOOKEEPER-2183.001.patch Tests use {{PortAssignment#unique}} for assignment of the ports to bind during tests. Currently, this method works by using a monotonically increasing counter from a static starting point. Generally, this is sufficient to achieve uniqueness within a single JVM process, but it does not achieve uniqueness across multiple processes on the same host. This can cause tests to get bind errors if there are multiple pre-commit jobs running concurrently on the same Jenkins host. This also prevents running tests in parallel to improve the speed of pre-commit runs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Failed: ZOOKEEPER-2183 PreCommit Build #2686
Jira: https://issues.apache.org/jira/browse/ZOOKEEPER-2183 Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2686/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 371868 lines...] [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 35 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] -1 core tests. The patch failed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2686//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2686//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2686//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] 8af7e65f85d95c6f25dabc7d5763a82127de1b32 logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] BUILD FAILED /home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-Build/trunk/build.xml:1732: exec returned: 1 Total time: 43 minutes 37 seconds Build step 'Execute shell' marked build as failure Archiving artifacts Sending artifact delta relative to PreCommit-ZOOKEEPER-Build #2685 Archived 7 artifacts Archive block size is 32768 Received 6 blocks and 366037 bytes Compression is 34.9% Took 4.8 sec Recording test results Description set: ZOOKEEPER-2183 Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## 2 tests failed. REGRESSION: org.apache.zookeeper.server.quorum.ReconfigRecoveryTest.testCurrentObserverIsParticipantInNewConfig Error Message: waiting for server 2 being up Stack Trace: junit.framework.AssertionFailedError: waiting for server 2 being up at org.apache.zookeeper.server.quorum.ReconfigRecoveryTest.testCurrentObserverIsParticipantInNewConfig(ReconfigRecoveryTest.java:529) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55) REGRESSION: org.apache.zookeeper.server.quorum.StandaloneDisabledTest.startSingleServerTest Error Message: null Stack Trace: junit.framework.AssertionFailedError at org.apache.zookeeper.test.ReconfigTest.testServerHasConfig(ReconfigTest.java:121) at org.apache.zookeeper.server.quorum.StandaloneDisabledTest.testReconfig(StandaloneDisabledTest.java:238) at org.apache.zookeeper.server.quorum.StandaloneDisabledTest.startSingleServerTest(StandaloneDisabledTest.java:91) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55)
[jira] [Commented] (ZOOKEEPER-2183) Change test port assignments to improve uniqueness of ports for multiple concurrent test processes on the same host.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539253#comment-14539253 ] Hongchao Deng commented on ZOOKEEPER-2183: -- StandaloneDisabledTest.startSingleServerTest was flaky in this case -- I run that single test successfully in local. Let me run again the entire test suite... Change test port assignments to improve uniqueness of ports for multiple concurrent test processes on the same host. Key: ZOOKEEPER-2183 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2183 Project: ZooKeeper Issue Type: Improvement Components: tests Reporter: Chris Nauroth Assignee: Chris Nauroth Attachments: ZOOKEEPER-2183.001.patch Tests use {{PortAssignment#unique}} for assignment of the ports to bind during tests. Currently, this method works by using a monotonically increasing counter from a static starting point. Generally, this is sufficient to achieve uniqueness within a single JVM process, but it does not achieve uniqueness across multiple processes on the same host. This can cause tests to get bind errors if there are multiple pre-commit jobs running concurrently on the same Jenkins host. This also prevents running tests in parallel to improve the speed of pre-commit runs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-2182) Several test suites are not running during pre-commit, because their names do not end with Test.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539278#comment-14539278 ] Rakesh R commented on ZOOKEEPER-2182: - Thanks all for taking care this. [~hdeng] am closing this jira as I think the changes has been applied/committed. Please feel to re-open if anything to be done. Several test suites are not running during pre-commit, because their names do not end with Test. -- Key: ZOOKEEPER-2182 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2182 Project: ZooKeeper Issue Type: Bug Components: tests Affects Versions: 3.5.0 Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 3.5.1, 3.6.0 Attachments: ZOOKEEPER-2182.001.patch In build.xml, the {{junit}} task definition uses an include pattern of {{\*\*/\*$\{test.category\}Test.java}}. This is important so that we don't accidentally try to run utility classes like {{PortAssignment}} or {{TestableZooKeeper}} as if they were JUnit suites. However, several test suites are misnamed so that they don't satisfy this pattern, and therefore pre-commit hasn't been running them. {{ClientRetry}} {{ReconfigFailureCases}} {{WatchEventWhenAutoReset}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-2182) Several test suites are not running during pre-commit, because their names do not end with Test.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539286#comment-14539286 ] Hongchao Deng commented on ZOOKEEPER-2182: -- Thank you! My mistake to forget close it. Several test suites are not running during pre-commit, because their names do not end with Test. -- Key: ZOOKEEPER-2182 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2182 Project: ZooKeeper Issue Type: Bug Components: tests Affects Versions: 3.5.0 Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 3.5.1, 3.6.0 Attachments: ZOOKEEPER-2182.001.patch In build.xml, the {{junit}} task definition uses an include pattern of {{\*\*/\*$\{test.category\}Test.java}}. This is important so that we don't accidentally try to run utility classes like {{PortAssignment}} or {{TestableZooKeeper}} as if they were JUnit suites. However, several test suites are misnamed so that they don't satisfy this pattern, and therefore pre-commit hasn't been running them. {{ClientRetry}} {{ReconfigFailureCases}} {{WatchEventWhenAutoReset}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-1667) Watch event isn't handled correctly when a client reestablish to a server
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539273#comment-14539273 ] Rakesh R commented on ZOOKEEPER-1667: - Good catch. Thanks [~cnauroth] Watch event isn't handled correctly when a client reestablish to a server - Key: ZOOKEEPER-1667 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1667 Project: ZooKeeper Issue Type: Bug Components: server Affects Versions: 3.3.6, 3.4.5 Reporter: Jacky007 Assignee: Flavio Junqueira Priority: Blocker Fix For: 3.4.6, 3.5.0 Attachments: ZOOKEEPER-1667-b3.4.patch, ZOOKEEPER-1667-b3.4.patch, ZOOKEEPER-1667-r34.patch, ZOOKEEPER-1667-trunk.patch, ZOOKEEPER-1667.patch When a client reestablish to a server, it will send the watches which have not been triggered. But the code in DataTree does not handle it correctly. It is obvious, we just do not notice it :) scenario: 1) Client a set a data watch on /d, then disconnect, client b delete /d and create it again. When client a reestablish to zk, it will receive a NodeCreated rather than a NodeDataChanged. 2) Client a set a exists watch on /e(not exist), then disconnect, client b create /e. When client a reestablish to zk, it will receive a NodeDataChanged rather than a NodeCreated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-2186) QuorumCnxManager#receiveConnection may crash with random input
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538294#comment-14538294 ] ASF GitHub Bot commented on ZOOKEEPER-2186: --- GitHub user rgs1 opened a pull request: https://github.com/apache/zookeeper/pull/30 ZOOKEEPER-2186: QuorumCnxManager#receiveConnection may crash with ran… …dom input You can merge this pull request into a Git repository by running: $ git pull https://github.com/rgs1/zookeeper ZOOKEEPER-2186 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/30.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #30 commit ff9cdc570fb4c84acf3c8b0c64875b5e54d82523 Author: Raul Gutierrez S r...@twitter.com Date: 2015-05-11T17:48:47Z ZOOKEEPER-2186: QuorumCnxManager#receiveConnection may crash with random input QuorumCnxManager#receiveConnection may crash with random input -- Key: ZOOKEEPER-2186 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2186 Project: ZooKeeper Issue Type: Bug Components: server Affects Versions: 3.4.6, 3.5.0 Reporter: Raul Gutierrez Segales Assignee: Raul Gutierrez Segales Fix For: 3.4.7, 3.5.1, 3.6.0 Attachments: ZOOKEEPER-2186.patch This will allocate an arbitrarily large byte buffer (and try to read it!): {code} public boolean receiveConnection(Socket sock) { Long sid = null; ... sid = din.readLong(); // next comes the #bytes in the remainder of the message int num_remaining_bytes = din.readInt(); byte[] b = new byte[num_remaining_bytes]; // remove the remainder of the message from din int num_read = din.read(b); {code} This will crash the QuorumCnxManager thread, so the cluster will keep going but future elections might fail to converge (ditto for leaving/joining members). Patch coming up in a bit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] zookeeper pull request: ZOOKEEPER-2186: QuorumCnxManager#receiveCo...
GitHub user rgs1 opened a pull request: https://github.com/apache/zookeeper/pull/30 ZOOKEEPER-2186: QuorumCnxManager#receiveConnection may crash with ran⦠â¦dom input You can merge this pull request into a Git repository by running: $ git pull https://github.com/rgs1/zookeeper ZOOKEEPER-2186 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/30.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #30 commit ff9cdc570fb4c84acf3c8b0c64875b5e54d82523 Author: Raul Gutierrez S r...@twitter.com Date: 2015-05-11T17:48:47Z ZOOKEEPER-2186: QuorumCnxManager#receiveConnection may crash with random input --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (ZOOKEEPER-2182) Several test suites are not running during pre-commit, because their names do not end with Test.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538257#comment-14538257 ] Chris Nauroth commented on ZOOKEEPER-2182: -- Thank you for the reviews and commit, Hongchao and Alex. Several test suites are not running during pre-commit, because their names do not end with Test. -- Key: ZOOKEEPER-2182 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2182 Project: ZooKeeper Issue Type: Bug Components: tests Affects Versions: 3.5.0 Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 3.5.1, 3.6.0 Attachments: ZOOKEEPER-2182.001.patch In build.xml, the {{junit}} task definition uses an include pattern of {{\*\*/\*$\{test.category\}Test.java}}. This is important so that we don't accidentally try to run utility classes like {{PortAssignment}} or {{TestableZooKeeper}} as if they were JUnit suites. However, several test suites are misnamed so that they don't satisfy this pattern, and therefore pre-commit hasn't been running them. {{ClientRetry}} {{ReconfigFailureCases}} {{WatchEventWhenAutoReset}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ZOOKEEPER-2186) QuorumCnxManager#receiveConnection may crash with random input
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538116#comment-14538116 ] Hongchao Deng commented on ZOOKEEPER-2186: -- [~rgs] Can you open a RB for this? I have some questions and comments to make. Thanks! QuorumCnxManager#receiveConnection may crash with random input -- Key: ZOOKEEPER-2186 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2186 Project: ZooKeeper Issue Type: Bug Components: server Affects Versions: 3.4.6, 3.5.0 Reporter: Raul Gutierrez Segales Assignee: Raul Gutierrez Segales Fix For: 3.4.7, 3.5.1, 3.6.0 Attachments: ZOOKEEPER-2186.patch This will allocate an arbitrarily large byte buffer (and try to read it!): {code} public boolean receiveConnection(Socket sock) { Long sid = null; ... sid = din.readLong(); // next comes the #bytes in the remainder of the message int num_remaining_bytes = din.readInt(); byte[] b = new byte[num_remaining_bytes]; // remove the remainder of the message from din int num_read = din.read(b); {code} This will crash the QuorumCnxManager thread, so the cluster will keep going but future elections might fail to converge (ditto for leaving/joining members). Patch coming up in a bit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)