Recently, the conversation <http://mail-archives.apache.org/mod_mbox/incubator-geode-dev/201507.mbox/browser> [0] came up about how do I "repeat" a test in the Geode test suite.
Invariably, the answer always centers around build-focused features. While it is unlikely that the build (i.e. *Gradle*) is going to change anytime soon, I'd rather see a solution built into the test framework used to test the codebase. Recently, I coded a (not-so-unique) solution <https://github.com/codeprimate-software/test-driven-development/blob/master/unit-testing/src/test/java/org/tdd/example/junit/RepeatingTestCasesExampleTest.java#L39-73> [1] to "repeating" a test case by implementing a *JUnit 4* Rule <https://github.com/junit-team/junit/wiki/Rules> [2] (RepeatRule <https://github.com/codeprimate-software/test-driven-development/blob/master/unit-testing/src/main/java/org/tdd/junit/rules/RepeatRule.java> [3]) and a @Repeat annotation <https://github.com/codeprimate-software/test-driven-development/blob/master/unit-testing/src/main/java/org/tdd/junit/Repeat.java> [4]. This was inspired from and implemented by other developers and frameworks (e.g. the *Spring Framework's* @Repeat <http://docs.spring.io/spring/docs/current/spring-framework-reference/htmlsingle/#integration-testing-annotations-junit> [5] test annotation). Unfortunately, the problem with most implementations is they require the use of a custom, *JUnit* Test Runner <https://github.com/junit-team/junit/wiki/Test-runners> [6]. The downside then is, you are unable to use other test runners for other purposes, such as using *JUnit's* Parameterized <https://github.com/junit-team/junit/wiki/Parameterized-tests> [7] tests, which requires the use of the Parameterized test runner, since the @RunWith annotation <https://github.com/junit-team/junit/blob/master/src/main/java/org/junit/runner/RunWith.java#L35> [8] only accepts a single test runner. Therefore, it is better to run as a *JUnit* Rule. To repeat a test, a developer just need annotate their test case method like so... @Test *@Repeat(10)* public void someTestCase() { ... } The @Repeat annotation accepts the a String value and defaults to 1 (run). The String type makes it handy to enable the use of System properties (with your own convention(s)) to control the number of runs from the command-line, using whatever build you desire (e.g. Gradle/Maven/Ant), or even from within your IDE, for example... Given... @Test *@Repeat("test.runs")* public void someTestCase() { .. } $ gradlew test -Dsingle.test=MyTestSuiteClass *-Dtest.runs=10* If the System property is not specified, the test run count defaults to 1. If the value is not "parseable" or "resolvable" (using a System property) as a numeric value, the test run count defaults to 1. The only downside to the Annotation approach is now you need to annotate each test case method with @Repeat annotation that you want to repeat. But, this is a 1 time thing and functions accordingly thereafter. If the community so desires, this solution can be integrated into the *Apache Geode* test codebase. I have filed GEODE-166 <https://issues.apache.org/jira/browse/GEODE-166> [9] with the attached source files for reference. I have limited time to perform this integration at the moment. Cheers, -- -John 503-504-8657 john.blum10101 (skype) [0] - http://mail-archives.apache.org/mod_mbox/incubator-geode-dev/201507.mbox/browser [1] - https://github.com/codeprimate-software/test-driven-development/blob/master/unit-testing/src/test/java/org/tdd/example/junit/RepeatingTestCasesExampleTest.java#L39-73 [2] - https://github.com/junit-team/junit/wiki/Rules [3] - https://github.com/codeprimate-software/test-driven-development/blob/master/unit-testing/src/main/java/org/tdd/junit/rules/RepeatRule.java [4] - https://github.com/codeprimate-software/test-driven-development/blob/master/unit-testing/src/main/java/org/tdd/junit/Repeat.java [5] - http://docs.spring.io/spring/docs/current/spring-framework-reference/htmlsingle/#integration-testing-annotations-junit [6] - https://github.com/junit-team/junit/wiki/Test-runners [7] - https://github.com/junit-team/junit/wiki/Parameterized-tests [8] - https://github.com/junit-team/junit/blob/master/src/main/java/org/junit/runner/RunWith.java#L35 [9] - https://issues.apache.org/jira/browse/GEODE-166
