[
https://issues.apache.org/jira/browse/SUREFIRE-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17735865#comment-17735865
]
Javier A. Ortiz commented on SUREFIRE-2180:
-------------------------------------------
Any hints on where I can find in the code where the invocations happen? I
assumed that it would be somewhere in the SureFire TestNG Runner module.
> Surefire not invoking tests with TestNG DataProvider
> ----------------------------------------------------
>
> Key: SUREFIRE-2180
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2180
> Project: Maven Surefire
> Issue Type: Bug
> Components: TestNG support
> Affects Versions: 3.1.2
> Reporter: Javier A. Ortiz
> Priority: Major
>
> I'm having a weird issue I can't figure out. It's driving me insane.
> I have a Maven project for which I'm running the same command both locally
> and on Jenkins. The environment is the same as far as I can tell:
> - Maven 3.9.2 (On Jenkins: Docker image maven:3.9.2-eclipse-temurin-11)
> - JDK 11 (On Jenkins: Docker image maven:3.9.2-eclipse-temurin-11)
> - TestNG 7.8.0
> Command:
> {code:java}
> mvn -Ddependency.surefire.verbose=10 -Pcoverage verify -B -U -Dgroup=PLATFORM
> {code}
> Relevant POM section:
> {code:java}
> <build>
> <pluginManagement>
> <plugins>
> <plugin>
> <groupId>org.jacoco</groupId>
> <artifactId>jacoco-maven-plugin</artifactId>
> <version>0.8.10</version>
> </plugin>
> ...
> </plugins>
> </pluginManagement>
> ...
> </build>
> ...
> <profiles>
> <profile>
> <id>coverage</id>
> <build>
> <plugins>
> <plugin>
> <groupId>org.jacoco</groupId>
> <artifactId>jacoco-maven-plugin</artifactId>
> <configuration>
> <excludes>
> <exclude>com/XXX/framework/dataproviders/**/*</exclude>
> <exclude>com/XXX/v2/basetest/**/*</exclude>
> <exclude>**/*Exception*</exclude>
> </excludes>
> </configuration>
> <executions>
> <execution>
> <goals>
> <goal>prepare-agent</goal>
> </goals>
> </execution>
> <!-- attached to Maven test phase -->
> <execution>
> <id>report</id>
> <phase>test</phase>
> <goals>
> <goal>report</goal>
> </goals>
> </execution>
> <!-- Add this checking -->
> <execution>
> <id>jacoco-check</id>
> <goals>
> <goal>check</goal>
> </goals>
> <configuration>
> <rules>
> <rule>
> <element>BUNDLE</element>
> <limits>
> <limit>
> <counter>INSTRUCTION</counter>
> <value>COVEREDRATIO</value>
> <minimum>60%</minimum>
> </limit>
> <limit>
> <counter>CLASS</counter>
> <value>MISSEDCOUNT</value>
> <maximum>0</maximum>
> </limit>
> </limits>
> </rule>
> <rule>
> <element>PACKAGE</element>
> <limits>
> <limit>
> <counter>LINE</counter>
> <value>COVEREDRATIO</value>
> <minimum>60%</minimum>
> </limit>
> </limits>
> </rule>
> </rules>
> <skip>${skipTests}</skip>
> </configuration>
> </execution>
> </executions>
> </plugin>
> </plugins>
> </build>
> </profile>
> {code}
> One of the tests that have the issue (only tests with DataProviders) show
> this pattern in the logs. Enabled verbose to level 10 in hopes that something
> would show up.
> Locally:
> {code:java}
> [2023-06-06 16:55:46.795] [INFO] [TestClass] Creating TestClass for
> [ClassImpl class=com.XXX.framework.utils.TimeUtilsTest]
> [2023-06-06 16:55:46.796] [INFO] Method public java.lang.Object[][]
> com.XXX.framework.utils.TimeUtilsTest.getTimeData() has a @Test annotation
> but also a return value: ignoring it. Use <suite allow-return-values="true">
> to fix this
> [2023-06-06 16:55:46.797] [INFO] [TestClass] Adding method
> TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0,
> instance:null] on TestClass class com.XXX.framework.utils.TimeUtilsTest
> {code}
> ...
> {code:java}
> [2023-06-06 16:55:47.316] [INFO] ===== Test class
> com.XXX.framework.utils.TimeUtilsTest
> [2023-06-06 16:55:47.316] [INFO] @Test
> TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0,
> instance:com.XXX.framework.utils.TimeUtilsTest@283eb984]
> [2023-06-06 16:55:47.316] [INFO] ======
> {code}
> ...
> {code:java}
> [TestNG] INVOKING: "Surefire test" -
> com.XXX.framework.utils.TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)(value(s):
> NANOSECONDS, 60, "60ns", TestNameParameter(customName=60 NANOSECONDS))
> {code}
> ...
> Test executes and results, etc.
> On Jenkins:
> {code:java}
> [2023-06-06 21:02:29.270] [INFO] [TestClass] Creating TestClass for
> [ClassImpl class=com.XXX.framework.utils.TimeUtilsTest]
> [2023-06-06 21:02:29.271] [INFO] Method public java.lang.Object[][]
> com.XXX.framework.utils.TimeUtilsTest.getTimeData() has a @Test annotation
> but also a return value: ignoring it. Use <suite allow-return-values="true">
> to fix this
> [2023-06-06 21:02:29.271] [INFO] [TestClass] Adding method
> TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0,
> instance:null] on TestClass class com.XXX.framework.utils.TimeUtilsTest
> {code}
> ...
> {code:java}
> [2023-06-06 21:02:29.529] [INFO] ===== Test class
> com.XXX.framework.utils.TimeUtilsTest
> [2023-06-06 21:02:29.529] [INFO] @Test
> TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0,
> instance:com.XXX.framework.utils.TimeUtilsTest@52f9a620]
> [2023-06-06 21:02:29.529] [INFO] ======
> {code}
> I see no other references about the test class in the rest of the trace.
> Basically, all tests with a DataProvider don't run.
> Maybe someone has a suggestion on what else to look for.
> I've tried the following:
> - Making sure the proper Test annotation is used (TestNG vs. JUnit
> - Remove test groups or any other listener that could be interfering.
> - Downgrade the TestNG version.
> - Pray.
> Here's the investigation on the TestNG side:
> [https://github.com/testng-team/testng/issues/2924]
> Question in StackOverflow:
> https://stackoverflow.com/questions/76418882/why-is-testng-not-running-tests-with-dataprovider
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
