Thanks Flavio for the pointer. Did some check today and find out: - The skipped test does not cause a build failure. In fact, this same test has been skipped for the past 70+ pre-commit builds, among which there are a couple of succeeded ones [1]. - The failure of this pre-commit build (3302) is caused by build.xml:1605, according to Jenkins. It is not obvious to me what failed from the log though. Will dig deeper. https://github.com/apache/zookeeper/blob/trunk/build.xml#L1605
Regarding the skipped test, it's QuorumTest::testSessionMove https://github.com/apache/zookeeper/blob/trunk/src/java/test/org/apache/zookeeper/test/QuorumTest.java#L253 This test (along with some others in QuorumTest) is ignored in https://issues.apache.org/jira/browse/ZOOKEEPER-733, because "session moved tests are currently ignored since they fail in netty. ". Then later, in https://issues.apache.org/jira/browse/ZOOKEEPER-907, all other ignored tests were activated, but not this one with no particular reason. I assume that just an overlook.. Also there is an existing JIRA about this skipped test, we should probably un-skip this test as it's legitimate. https://issues.apache.org/jira/browse/ZOOKEEPER-2285 [1] Succeeded pre-commit build with testSessionMove test skipped: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3308/ https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3290/ On Fri, Jul 29, 2016 at 2:06 AM, Flavio Junqueira <f...@apache.org> wrote: > There seems to be something wrong with this code: > > public static List<FrameworkMethod> computeTestMethodsForClass(final Class > klass, final List<FrameworkMethod> defaultMethods) { > List<FrameworkMethod> list = defaultMethods; > String methodName = System.getProperty("test.method"); > if (methodName == null) { > LOG.info("No test.method specified. using default methods."); > } else { > LOG.info("Picked up test.method={}", methodName); > try { > list = Arrays.asList(new > FrameworkMethod(klass.getMethod(methodName))); > } catch (NoSuchMethodException nsme) { > LOG.warn("{} does not have test.method={}. failing to > default methods.", klass.getName(), methodName); > } > } > return list; > } > > This is in JUnit4ZKTestRunner. Perhaps a race? > > -Flavio > > > On 29 Jul 2016, at 05:12, Michael Han <h...@cloudera.com> wrote: > > > > While triage Jenkins builds, I find out that some tests could be in the > > state of 'Skipped'. This build is an example, in particular this is the > > skipped test > > > https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302/testReport/org.apache.zookeeper.test/QuorumTest/testSessionMove/ > > > > It looks like if there is any test that's in 'skipped' state, then the > > build will be marked as failure, regardless if all other tests passed or > > not. > > Do we know what could cause a test being skipped? The log said 'No > > test.method specified.' but this does not deterministically happen for > any > > specific tests. > > > > > > Here is the log of the skipped test: > > Skipped > > > > org.apache.zookeeper.test.QuorumTest.testSessionMove > > Skipped for the past 82 builds (Since > > <https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/>#3221 ) > > Took 1 ms. > > < > https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302/testReport/org.apache.zookeeper.test/QuorumTest/testSessionMove/history > > > > Standard Output > > > > 2016-07-29 01:06:54,754 [myid:] - INFO [main:JUnit4ZKTestRunner@47] - > > No test.method specified. using default methods. > > 2016-07-29 01:06:54,813 [myid:] - INFO [main:JUnit4ZKTestRunner@47] - > > No test.method specified. using default methods. > > 2016-07-29 01:06:54,827 [myid:] - INFO [main:PortAssignment@151] - > > Test process 4/8 using ports from 19300 - 21992. > > 2016-07-29 01:06:54,829 [myid:] - INFO [main:PortAssignment@85] - > > Assigned port 19301 from range 19300 - 21992. > > 2016-07-29 01:06:54,835 [myid: > > ...[truncated 2825034 chars]... > > il@243] - 127.0.0.1:19566 is no longer accepting client connections > > 2016-07-29 01:09:44,398 [myid:] - INFO [main:FourLetterWordMain@85] - > > connecting to 127.0.0.1 19569 > > 2016-07-29 01:09:44,398 [myid:] - INFO [main:QuorumUtil@243] - > > 127.0.0.1:19569 is no longer accepting client connections > > 2016-07-29 01:09:44,399 [myid:] - INFO [main:ZKTestCase$1@65] - > > SUCCEEDED testFollowersStartAfterLeader > > 2016-07-29 01:09:44,399 [myid:] - INFO [main:ZKTestCase$1@60] - > > FINISHED testFollowersStartAfterLeader > > > > > > > > > > On Thu, Jul 28, 2016 at 6:20 PM, Apache Jenkins Server < > > jenk...@builds.apache.org> wrote: > > > >> Jira: https://issues.apache.org/jira/browse/ZOOKEEPER-2492 > >> Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302/ > >> > >> > >> > ################################################################################### > >> ########################## LAST 60 LINES OF THE CONSOLE > >> ########################### > >> [...truncated 443101 lines...] > >> [exec] -1 tests included. The patch doesn't appear to include > >> any new or modified tests. > >> [exec] Please justify why no new tests are > >> needed for this patch. > >> [exec] Also please list what manual steps > >> were performed to verify this patch. > >> [exec] > >> [exec] +1 javadoc. The javadoc tool did not generate any > warning > >> messages. > >> [exec] > >> [exec] +1 javac. The applied patch does not increase the total > >> number of javac compiler warnings. > >> [exec] > >> [exec] +1 findbugs. The patch does not introduce any new > >> Findbugs (version 2.0.3) warnings. > >> [exec] > >> [exec] +1 release audit. The applied patch does not increase > the > >> total number of release audit warnings. > >> [exec] > >> [exec] +1 core tests. The patch passed core unit tests. > >> [exec] > >> [exec] +1 contrib tests. The patch passed contrib unit tests. > >> [exec] > >> [exec] Test results: > >> > https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302//testReport/ > >> [exec] Findbugs warnings: > >> > https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html > >> [exec] Console output: > >> https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302//console > >> [exec] > >> [exec] This message is automatically generated. > >> [exec] > >> [exec] > >> [exec] > >> ====================================================================== > >> [exec] > >> ====================================================================== > >> [exec] Adding comment to Jira. > >> [exec] > >> ====================================================================== > >> [exec] > >> ====================================================================== > >> [exec] > >> [exec] > >> [exec] Comment added. > >> [exec] 69951929bc7887930c177e2a8617f3c2a0800e52 logged out > >> [exec] > >> [exec] > >> [exec] > >> ====================================================================== > >> [exec] > >> ====================================================================== > >> [exec] Finished build. > >> [exec] > >> ====================================================================== > >> [exec] > >> ====================================================================== > >> [exec] > >> [exec] > >> > >> BUILD FAILED > >> > /home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-Build/trunk/build.xml:1605: > >> exec returned: 1 > >> > >> Total time: 21 minutes 17 seconds > >> Build step 'Execute shell' marked build as failure > >> Archiving artifacts > >> Setting > >> > LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7 > >> Recording test results > >> Setting > >> > LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7 > >> Setting > >> > LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7 > >> [description-setter] Description set: ZOOKEEPER-2492 > >> Email was triggered for: Failure - Any > >> Sending email for trigger: Failure - Any > >> Setting > >> > LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7 > >> Setting > >> > LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7 > >> Setting > >> > LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7 > >> Setting > >> > LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7 > >> > >> > >> > >> > >> > ################################################################################### > >> ############################## FAILED TESTS (if any) > >> ############################## > >> All tests passed > > > > > > > > > > -- > > Cheers > > Michael. > > -- Cheers Michael.
