[
https://issues.apache.org/jira/browse/SUREFIRE-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15141855#comment-15141855
]
Tibor Digana edited comment on SUREFIRE-1228 at 2/10/16 10:49 PM:
--
[~cmorris_partnet]
rerunFailingTestsCount is applicable only to {{surefire-junit4}} or
{{surefire-junit47}} provider.
As soon as you specify TestNG dependency in POM then {{surefire-testng}}
provider is activated and the rerun is not applicable.
was (Author: tibor17):
[~cmorris_partnet]
rerunFailingTestsCount is applicable only to {{surefire-junit4}} or
{{surefire-junit47}} provider.
As soon as you specify TestNG dependency in POM then {{surefire-testng}}
provider is activated and the rerun is not applicable.
Do you need more help?
> rerunFailingTestsCount + TestNG + @DataProvider = failed tests pass
> ---
>
> Key: SUREFIRE-1228
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1228
> Project: Maven Surefire
> Issue Type: Bug
> Components: TestNG support
>Affects Versions: 2.19.1
>Reporter: Cam Morris
> Attachments: TestRandomFail.java, pom.xml
>
>
> Surefire confuses tests run with a data-provider with rerun tests so only one
> of the data-provided tests need to pass.
> Steps to reproduce:
> # Create a testNG test with a data provider that passes with some of the data
> and fails with others
> {code:title=TestNG example|language=java}
> @Test(dataProvider = "succeed")
> public void fail(boolean succeed) {
> if (!succeed) {
> Assert.fail("nope");
> }
> }
> @DataProvider(name="succeed")
> public Object[][] arrayIndex() {
> return new Object[][]{ {true}, {false}};
> }
> {code}
> # Configure surefire to rerun failed tests
> {code:title=pom.xml example|language=xml}
>
> http://maven.apache.org/POM/4.0.0";
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> http://maven.apache.org/xsd/maven-4.0.0.xsd";>
> 4.0.0
> cam.test
> testng-retry
> 1.0-SNAPSHOT
>
>
> org.testng
> testng
> 6.9.10
> test
>
>
>
>
>
> org.apache.maven.plugins
> maven-surefire-plugin
> 2.19.1
>
> 1
>
>
>
>
>
> {code}
> - *Expected results*: "Tests run: 2, Failures: 1..." The tests should not
> pass.
> - *Actual results*: "Tests run: 1, Failures: 0, Errors: 0, Skipped: 0,
> Flakes: 1" The rerun confuses test runs with different data for retry
> attempts. Just one of the data provider tests needs to succeed for the test
> to be deemed a success.
> {code}mvn test
> ...
> ---
> T E S T S
> ---
> Running TestRandomFail
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.267 sec <<<
> FAILURE! - in TestRandomFail
> fail(TestRandomFail) Time elapsed: 0.007 sec <<< FAILURE!
> java.lang.AssertionError: nope
> at TestRandomFail.fail(TestRandomFail.java:14)
> Results :
> Flaked tests:
> TestRandomFail.fail(TestRandomFail)
> Run 1: PASS
> Run 2: TestRandomFail.fail:14 nope
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Flakes: 1
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 2.251 s
> [INFO] Finished at: 2016-02-10T07:39:21-07:00
> [INFO] Final Memory: 9M/244M
> [INFO]
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
