There seems to be something wrong with this code:
public static List<FrameworkMethod> computeTestMethodsForClass(final Class
klass, final List<FrameworkMethod> defaultMethods) {
List<FrameworkMethod> list = defaultMethods;
String methodName = System.getProperty("test.method");
if (methodName == null) {
LOG.info("No test.method specified. using default methods.");
} else {
LOG.info("Picked up test.method={}", methodName);
try {
list = Arrays.asList(new
FrameworkMethod(klass.getMethod(methodName)));
} catch (NoSuchMethodException nsme) {
LOG.warn("{} does not have test.method={}. failing to default
methods.", klass.getName(), methodName);
}
}
return list;
}
This is in JUnit4ZKTestRunner. Perhaps a race?
-Flavio
> On 29 Jul 2016, at 05:12, Michael Han <[email protected]> wrote:
>
> While triage Jenkins builds, I find out that some tests could be in the
> state of 'Skipped'. This build is an example, in particular this is the
> skipped test
> https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302/testReport/org.apache.zookeeper.test/QuorumTest/testSessionMove/
>
> It looks like if there is any test that's in 'skipped' state, then the
> build will be marked as failure, regardless if all other tests passed or
> not.
> Do we know what could cause a test being skipped? The log said 'No
> test.method specified.' but this does not deterministically happen for any
> specific tests.
>
>
> Here is the log of the skipped test:
> Skipped
>
> org.apache.zookeeper.test.QuorumTest.testSessionMove
> Skipped for the past 82 builds (Since
> <https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/>#3221 )
> Took 1 ms.
> <https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302/testReport/org.apache.zookeeper.test/QuorumTest/testSessionMove/history>
> Standard Output
>
> 2016-07-29 01:06:54,754 [myid:] - INFO [main:JUnit4ZKTestRunner@47] -
> No test.method specified. using default methods.
> 2016-07-29 01:06:54,813 [myid:] - INFO [main:JUnit4ZKTestRunner@47] -
> No test.method specified. using default methods.
> 2016-07-29 01:06:54,827 [myid:] - INFO [main:PortAssignment@151] -
> Test process 4/8 using ports from 19300 - 21992.
> 2016-07-29 01:06:54,829 [myid:] - INFO [main:PortAssignment@85] -
> Assigned port 19301 from range 19300 - 21992.
> 2016-07-29 01:06:54,835 [myid:
> ...[truncated 2825034 chars]...
> il@243] - 127.0.0.1:19566 is no longer accepting client connections
> 2016-07-29 01:09:44,398 [myid:] - INFO [main:FourLetterWordMain@85] -
> connecting to 127.0.0.1 19569
> 2016-07-29 01:09:44,398 [myid:] - INFO [main:QuorumUtil@243] -
> 127.0.0.1:19569 is no longer accepting client connections
> 2016-07-29 01:09:44,399 [myid:] - INFO [main:ZKTestCase$1@65] -
> SUCCEEDED testFollowersStartAfterLeader
> 2016-07-29 01:09:44,399 [myid:] - INFO [main:ZKTestCase$1@60] -
> FINISHED testFollowersStartAfterLeader
>
>
>
>
> On Thu, Jul 28, 2016 at 6:20 PM, Apache Jenkins Server <
> [email protected]> wrote:
>
>> Jira: https://issues.apache.org/jira/browse/ZOOKEEPER-2492
>> Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302/
>>
>>
>> ###################################################################################
>> ########################## LAST 60 LINES OF THE CONSOLE
>> ###########################
>> [...truncated 443101 lines...]
>> [exec] -1 tests included. The patch doesn't appear to include
>> any new or modified tests.
>> [exec] Please justify why no new tests are
>> needed for this patch.
>> [exec] Also please list what manual steps
>> were performed to verify this patch.
>> [exec]
>> [exec] +1 javadoc. The javadoc tool did not generate any warning
>> messages.
>> [exec]
>> [exec] +1 javac. The applied patch does not increase the total
>> number of javac compiler warnings.
>> [exec]
>> [exec] +1 findbugs. The patch does not introduce any new
>> Findbugs (version 2.0.3) warnings.
>> [exec]
>> [exec] +1 release audit. The applied patch does not increase the
>> total number of release audit warnings.
>> [exec]
>> [exec] +1 core tests. The patch passed core unit tests.
>> [exec]
>> [exec] +1 contrib tests. The patch passed contrib unit tests.
>> [exec]
>> [exec] Test results:
>> https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302//testReport/
>> [exec] Findbugs warnings:
>> https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
>> [exec] Console output:
>> https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3302//console
>> [exec]
>> [exec] This message is automatically generated.
>> [exec]
>> [exec]
>> [exec]
>> ======================================================================
>> [exec]
>> ======================================================================
>> [exec] Adding comment to Jira.
>> [exec]
>> ======================================================================
>> [exec]
>> ======================================================================
>> [exec]
>> [exec]
>> [exec] Comment added.
>> [exec] 69951929bc7887930c177e2a8617f3c2a0800e52 logged out
>> [exec]
>> [exec]
>> [exec]
>> ======================================================================
>> [exec]
>> ======================================================================
>> [exec] Finished build.
>> [exec]
>> ======================================================================
>> [exec]
>> ======================================================================
>> [exec]
>> [exec]
>>
>> BUILD FAILED
>> /home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-Build/trunk/build.xml:1605:
>> exec returned: 1
>>
>> Total time: 21 minutes 17 seconds
>> Build step 'Execute shell' marked build as failure
>> Archiving artifacts
>> Setting
>> LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7
>> Recording test results
>> Setting
>> LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7
>> Setting
>> LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7
>> [description-setter] Description set: ZOOKEEPER-2492
>> Email was triggered for: Failure - Any
>> Sending email for trigger: Failure - Any
>> Setting
>> LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7
>> Setting
>> LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7
>> Setting
>> LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7
>> Setting
>> LATEST1_7_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7
>>
>>
>>
>>
>> ###################################################################################
>> ############################## FAILED TESTS (if any)
>> ##############################
>> All tests passed
>
>
>
>
> --
> Cheers
> Michael.
