[ https://issues.apache.org/jira/browse/SUREFIRE-1219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylwester Lachiewicz reassigned SUREFIRE-1219: ---------------------------------------------- Assignee: (was: Tibor Digana) > skipAfterFailureCount should not count flaky tests as failures when using > rerunFailingTestsCount > ------------------------------------------------------------------------------------------------ > > Key: SUREFIRE-1219 > URL: https://issues.apache.org/jira/browse/SUREFIRE-1219 > Project: Maven Surefire > Issue Type: Bug > Affects Versions: 2.19.1 > Reporter: Sean Flanigan > Priority: Minor > Attachments: SUREFIRE-1219-tests.zip > > > According to > https://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html > "failed tests within re-run phase are not included in > skipAfterFailureCount". For instance, if skipAfterFailureCount is 1, but > every test passes on the second attempt, the tests are "flaky" but not > "failures". This should be reflected in summaries, in XML result files and > in fail fast (skipAfterFailureCount), but this is not the case. > If I have 10 flaky-but-not-failing tests, set rerunFailingTestsCount=1 and > skipAfterFailureCount=5: > 1. I would expect something like this to be printed on the console: > Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Flakes: 10 > 2. The XML report should list <flakyFailure>s but no <failures>. > 3. All tests should be run, not just 5 of them, in other words > skipAfterFailureCount should not count the flaky tests as part of the failure > count. > I have some tests demonstrating the problem, which I will attach to this > issue. -- This message was sent by Atlassian Jira (v8.20.10#820010)