[ https://issues.apache.org/jira/browse/SUREFIRE-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733531#comment-17733531 ]
Gerold Broser commented on SUREFIRE-2180: ----------------------------------------- I recently learned at SO that Surefire seems to be a bit picky regarding the (correct–at least what it "thinks" is the correct) version of the test framework dependencies declared in the POM. See the very end of my answer to [_Java unit test run successfully on Intellij but fail on maven_](https://stackoverflow.com/a/76426292/1744774): {quote}So, it seems that incompatibilities between JUnit dependencies (in a certain version) and Maven's Surefire plugin (in a certain version) might have caused the original issue.{quote} Perhaps something similar applies to this issue as well. > Surefire not invoking tests with TestNG DataProvider > ---------------------------------------------------- > > Key: SUREFIRE-2180 > URL: https://issues.apache.org/jira/browse/SUREFIRE-2180 > Project: Maven Surefire > Issue Type: Bug > Components: TestNG support > Affects Versions: 3.1.2 > Reporter: Javier A. Ortiz > Priority: Major > > I'm having a weird issue I can't figure out. It's driving me insane. > I have a Maven project for which I'm running the same command both locally > and on Jenkins. The environment is the same as far as I can tell: > - Maven 3.9.2 (On Jenkins: Docker image maven:3.9.2-eclipse-temurin-11) > - JDK 11 (On Jenkins: Docker image maven:3.9.2-eclipse-temurin-11) > - TestNG 7.8.0 > Command: > {code:java} > mvn -Ddependency.surefire.verbose=10 -Pcoverage verify -B -U -Dgroup=PLATFORM > {code} > Relevant POM section: > {code:java} > <build> > <pluginManagement> > <plugins> > <plugin> > <groupId>org.jacoco</groupId> > <artifactId>jacoco-maven-plugin</artifactId> > <version>0.8.10</version> > </plugin> > ... > </plugins> > </pluginManagement> > ... > </build> > ... > <profiles> > <profile> > <id>coverage</id> > <build> > <plugins> > <plugin> > <groupId>org.jacoco</groupId> > <artifactId>jacoco-maven-plugin</artifactId> > <configuration> > <excludes> > <exclude>com/XXX/framework/dataproviders/**/*</exclude> > <exclude>com/XXX/v2/basetest/**/*</exclude> > <exclude>**/*Exception*</exclude> > </excludes> > </configuration> > <executions> > <execution> > <goals> > <goal>prepare-agent</goal> > </goals> > </execution> > <!-- attached to Maven test phase --> > <execution> > <id>report</id> > <phase>test</phase> > <goals> > <goal>report</goal> > </goals> > </execution> > <!-- Add this checking --> > <execution> > <id>jacoco-check</id> > <goals> > <goal>check</goal> > </goals> > <configuration> > <rules> > <rule> > <element>BUNDLE</element> > <limits> > <limit> > <counter>INSTRUCTION</counter> > <value>COVEREDRATIO</value> > <minimum>60%</minimum> > </limit> > <limit> > <counter>CLASS</counter> > <value>MISSEDCOUNT</value> > <maximum>0</maximum> > </limit> > </limits> > </rule> > <rule> > <element>PACKAGE</element> > <limits> > <limit> > <counter>LINE</counter> > <value>COVEREDRATIO</value> > <minimum>60%</minimum> > </limit> > </limits> > </rule> > </rules> > <skip>${skipTests}</skip> > </configuration> > </execution> > </executions> > </plugin> > </plugins> > </build> > </profile> > {code} > One of the tests that have the issue (only tests with DataProviders) show > this pattern in the logs. Enabled verbose to level 10 in hopes that something > would show up. > Locally: > {code:java} > [2023-06-06 16:55:46.795] [INFO] [TestClass] Creating TestClass for > [ClassImpl class=com.XXX.framework.utils.TimeUtilsTest] > [2023-06-06 16:55:46.796] [INFO] Method public java.lang.Object[][] > com.XXX.framework.utils.TimeUtilsTest.getTimeData() has a @Test annotation > but also a return value: ignoring it. Use <suite allow-return-values="true"> > to fix this > [2023-06-06 16:55:46.797] [INFO] [TestClass] Adding method > TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0, > instance:null] on TestClass class com.XXX.framework.utils.TimeUtilsTest > {code} > ... > {code:java} > [2023-06-06 16:55:47.316] [INFO] ===== Test class > com.XXX.framework.utils.TimeUtilsTest > [2023-06-06 16:55:47.316] [INFO] @Test > TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0, > instance:com.XXX.framework.utils.TimeUtilsTest@283eb984] > [2023-06-06 16:55:47.316] [INFO] ====== > {code} > ... > {code:java} > [TestNG] INVOKING: "Surefire test" - > com.XXX.framework.utils.TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)(value(s): > NANOSECONDS, 60, "60ns", TestNameParameter(customName=60 NANOSECONDS)) > {code} > ... > Test executes and results, etc. > On Jenkins: > {code:java} > [2023-06-06 21:02:29.270] [INFO] [TestClass] Creating TestClass for > [ClassImpl class=com.XXX.framework.utils.TimeUtilsTest] > [2023-06-06 21:02:29.271] [INFO] Method public java.lang.Object[][] > com.XXX.framework.utils.TimeUtilsTest.getTimeData() has a @Test annotation > but also a return value: ignoring it. Use <suite allow-return-values="true"> > to fix this > [2023-06-06 21:02:29.271] [INFO] [TestClass] Adding method > TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0, > instance:null] on TestClass class com.XXX.framework.utils.TimeUtilsTest > {code} > ... > {code:java} > [2023-06-06 21:02:29.529] [INFO] ===== Test class > com.XXX.framework.utils.TimeUtilsTest > [2023-06-06 21:02:29.529] [INFO] @Test > TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0, > instance:com.XXX.framework.utils.TimeUtilsTest@52f9a620] > [2023-06-06 21:02:29.529] [INFO] ====== > {code} > I see no other references about the test class in the rest of the trace. > Basically, all tests with a DataProvider don't run. > Maybe someone has a suggestion on what else to look for. > I've tried the following: > - Making sure the proper Test annotation is used (TestNG vs. JUnit > - Remove test groups or any other listener that could be interfering. > - Downgrade the TestNG version. > - Pray. > Here's the investigation on the TestNG side: > [https://github.com/testng-team/testng/issues/2924] > Question in StackOverflow: > https://stackoverflow.com/questions/76418882/why-is-testng-not-running-tests-with-dataprovider -- This message was sent by Atlassian Jira (v8.20.10#820010)