[
https://issues.apache.org/jira/browse/SUREFIRE-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16418948#comment-16418948
]
Tibor Digana commented on SUREFIRE-1470:
----------------------------------------
[~alexey.subach]
The groups are not handled by {{Junit4Provider}} means {{surefire-junit4}}
provider.
See the Feature Matrix:
http://maven.apache.org/surefire/maven-surefire-plugin/featurematrix.html
Your experience is, I think, misunderstood because the provider
surefire-junit47 is activated together with groups; and opposite when
surefire-junit4 is activated without groups. The rerun feature exists in both
and therefore you may see different results.
> Unexpected report result when using rerunFailingTestsCount with groups filter
> in Surefire
> -----------------------------------------------------------------------------------------
>
> Key: SUREFIRE-1470
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1470
> Project: Maven Surefire
> Issue Type: Bug
> Components: Junit 4.x support, Maven Surefire Plugin
> Reporter: Alexey Subach
> Priority: Major
> Attachments: test.zip
>
>
> When the following plugin configuration is used:
> {code:java}
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-surefire-plugin</artifactId>
> <version>${surefire.version}</version>
> <configuration>
> <includes>
> <include>**/*.java</include>
> </includes>
> <groups>TestGroup</groups>
> <rerunFailingTestsCount>1</rerunFailingTestsCount>
> </configuration>
> </plugin>{code}
> And there is a test that fails first time it's run, the resultant *txt*
> report is unexpected and only contains the result of last rerun.
> Suppose we have the following test class:
> {code:java}
> @Category(TestGroup.class)
> public class CustomTest {
> @Test
> public void alwaysPassingTest() {
> Assert.assertEquals(0, 0);
> }
> private static volatile int x = 0;
> @Test
> public synchronized void secondAttemptPassingTest() throws
> InterruptedException {
> Thread.sleep(5000);
> if (x++ == 0) {
> Assert.fail();
> }
> }
> }
> {code}
> The resultant xml report is all right, but the txt one is
> {code}
> -------------------------------------------------------------------------------
> Test set: CustomTest
> -------------------------------------------------------------------------------
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 s - in
> CustomTest
> {code}
> Which is basically incorrect. The *txt* file is regenerated several times
> during the test run.
> The issue is present when groups filter (either exclusion or inclusion) is
> present. When there is no group filter and only rerunFailingTestsCount is
> configured, then the resultant report is the following one:
> {code}
> -------------------------------------------------------------------------------
> Test set: CustomTest
> -------------------------------------------------------------------------------
> Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.065 s <<<
> FAILURE! - in CustomTest
> secondAttemptPassingTest(CustomTest) Time elapsed: 0.009 s <<< FAILURE!
> java.lang.AssertionError
> at CustomTest.secondAttemptPassingTest(CustomTest.java:18)
> {code}
> When no group filter is configured, Junit4Provider instead of JunitCore
> provider is picked up and they seem to handle those cases differently.
> The situation is worsened by the fact that TeamCity is not able to parse the
> results correctly when the groups are configured along with
> rerunFailingTestsCount and it marks the tests that passed after rerun as
> failed, although the overall Maven result is SUCCESS in both cases.
> The simple Maven project to reproduce the issue is attached.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
