[
https://issues.apache.org/jira/browse/SUREFIRE-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082420#comment-17082420
]
Elliotte Rusty Harold commented on SUREFIRE-1638:
-------------------------------------------------
It's a useful idea, though the name could be much better. Perhaps retryCount?
> untilFailureLoopCount property to run tests until failure
> ---------------------------------------------------------
>
> Key: SUREFIRE-1638
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1638
> Project: Maven Surefire
> Issue Type: New Feature
> Reporter: Galder Zamarreño
> Priority: Major
>
> This is a proposal to add surefire/failsafe property called
> `untilFailureLoopCount`.
> I often find myself in the situation where there's a random test failure
> happening in CI, and have to run the one or several tests continuously until
> it fails, maybe running them with TRACE logging.
> Normally, I'd wrap the `mvn test...` call with a script that calls it in a
> loop, but this can be quite "slow" since each iteration is a new JVM
> launch...etc.
> The idea behind `untilFailureLoopCount` is that you've give it a positive
> number, say 1000, and the test(s) would run continuously until either a
> failure happens or the test is executed that many times.
> I've created [this randomly failing
> test](https://github.com/galderz/surefire-until-failure-loop-count/blob/master/src/test/java/com/acme/maven/surefire/untilfailure/UntilFailureLoopCountTest.java)
> and I've run it with `mvn -Dsurefire.untilFailureLoopCount=10 test` and the
> output would be something like:
> {code}
> [INFO] -------------------------------------------------------
> [INFO] T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.034
> s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001
> s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s -
> in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s -
> in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.001
> s <<< FAILURE! - in
> com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [ERROR]
> com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest.testEventualFailure
> Time elapsed: 0.001 s <<< FAILURE!
> java.lang.AssertionError: Just messing with your testing
> at
> com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest.testEventualFailure(UntilFailureLoopCountTest.java:16)
> {code}
> If maintainers are interested in this, I can complete the PR with tests,
> documentation and support for other providers.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
