This is an automated email from the ASF dual-hosted git repository.
olamy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/maven-surefire.git
The following commit(s) were added to refs/heads/master by this push:
new c1ebd7f02 Discover each test class on its own again (#3572)
c1ebd7f02 is described below
commit c1ebd7f021097d7faa96c4c15178361894118504
Author: Olivier Lamy <[email protected]>
AuthorDate: Thu Nov 12 21:15:19 2026 +1000
Discover each test class on its own again (#3572)
Since 3.6.1 the scan discovered all test classes in one go and matched
the result back to class names. Engines don't have to report tests under
the name of the class you picked, so some classes were dropped and their
tests silently skipped. A nested @ParameterizedClass picked with -Dtest
is one example.
This reverts a35e23b. The catch is that TestNG suite listeners get called
once per scanned class during the scan again, like in 3.6.0. The TestNG
page now shows how a listener can skip the engine's dry runs.
Fixes #3548
Signed-off-by: Olivier Lamy <[email protected]>
---
.../src/site/markdown/examples/testng.md.vm | 26 +++
.../src/site/markdown/whats-new-3-6-x.md | 2 +
.../jiras/Surefire3462TestNgSuiteListenerIT.java | 69 -------
.../Surefire3548NestedParameterizedClassIT.java} | 22 ++-
.../resources/surefire-3462-testng-only/pom.xml | 66 -------
.../src/test/java/issue3462/BaseTest.java | 30 ---
.../src/test/java/issue3462/TestNGTest.java | 31 ---
.../test/java/issue3462/TrackingSuiteListener.java | 38 ----
.../META-INF/services/org.testng.ITestNGListener | 1 -
.../surefire-3462-testng-suite-listener/pom.xml | 72 -------
.../test/java/issue3462/TrackingSuiteListener.java | 39 ----
.../META-INF/services/org.testng.ITestNGListener | 1 -
.../pom.xml | 60 ++++++
.../CalculatorParameterizedClassTests.java} | 34 +++-
.../junitplatform/JUnitPlatformProvider.java | 57 +-----
.../junitplatform/TestNgTestClassDetector.java | 161 ----------------
.../junitplatform/TestPlanScannerFilter.java | 142 ++------------
.../junitplatform/TestNgTestClassDetectorTest.java | 150 ---------------
.../junitplatform/TestPlanScannerFilterTest.java | 211 ++++-----------------
19 files changed, 194 insertions(+), 1018 deletions(-)
diff --git a/maven-surefire-plugin/src/site/markdown/examples/testng.md.vm
b/maven-surefire-plugin/src/site/markdown/examples/testng.md.vm
index 883ec2f19..feceefa8d 100644
--- a/maven-surefire-plugin/src/site/markdown/examples/testng.md.vm
+++ b/maven-surefire-plugin/src/site/markdown/examples/testng.md.vm
@@ -202,6 +202,32 @@ For more information on TestNG, see the [TestNG web
site](https://testng.org).
You can implement TestNG listener interface `org.testng.ITestListener` in a
separate test artifact `your-testng-listener-artifact` with scope=test, or in
project test source code `src/test/java`. You can filter test artifacts by the
parameter `dependenciesToScan` to load its classes in current ClassLoader of
the test provider. The TestNG reporter class should implement
`org.testng.IReporter`.
+Listeners and the TestNG dry run
+--------------------------------
+
+Since 3.6.0, TestNG tests run on the JUnit Platform. To find the tests, the
TestNG engine runs TestNG in
+dry-run mode, and a dry run calls your listeners just like a real run.
Surefire asks the engine about each
+test class while it scans them, then once more right before the tests run, so
`ISuiteListener.onStart()`
+and `onFinish()` are called more than once. The engine also dry-runs classes
that are not TestNG tests,
+so this happens as soon as TestNG is on the test classpath, even in a module
with only JUnit tests.
+
+During a dry run the system property `testng.mode.dryrun` is `true`. A
listener that does real work, like
+starting a server, should skip it then:
+
+```java
+public class MySuiteListener implements ISuiteListener {
+ @Override
+ public void onStart(ISuite suite) {
+ if (Boolean.getBoolean("testng.mode.dryrun")) {
+ return;
+ }
+ // the real setup goes here
+ }
+}
+```
+
+See
[testng-engine#440](https://github.com/junit-team/testng-engine/issues/440) for
the background.
+
The level of verbosity
----------------------
diff --git a/maven-surefire-plugin/src/site/markdown/whats-new-3-6-x.md
b/maven-surefire-plugin/src/site/markdown/whats-new-3-6-x.md
index f64c00f75..a05ffc7cc 100644
--- a/maven-surefire-plugin/src/site/markdown/whats-new-3-6-x.md
+++ b/maven-surefire-plugin/src/site/markdown/whats-new-3-6-x.md
@@ -205,6 +205,8 @@ Ensure your TestNG version is **6.14.3 or later**. Surefire
will automatically u
TestNG configuration (groups, listeners, reporters, parallel, etc.) continues
to work through Surefire's plugin
configuration and is mapped to the JUnit Platform infrastructure.
+Listeners are also called during the TestNG engine's dry runs, see
+[Listeners and the TestNG dry
run](examples/testng.html#Listeners_and_the_TestNG_dry_run).
TestNG suite XML files (`suiteXmlFiles`) are no longer supported, and since
3.6.1 the build fails if they're set.
Pick your tests with `groups`, `includes`/`excludes` or a JUnit Platform
`@Suite` class instead.
diff --git
a/surefire-its/src/test/java/org/apache/maven/surefire/its/jiras/Surefire3462TestNgSuiteListenerIT.java
b/surefire-its/src/test/java/org/apache/maven/surefire/its/jiras/Surefire3462TestNgSuiteListenerIT.java
deleted file mode 100644
index f8c1f867d..000000000
---
a/surefire-its/src/test/java/org/apache/maven/surefire/its/jiras/Surefire3462TestNgSuiteListenerIT.java
+++ /dev/null
@@ -1,69 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-package org.apache.maven.surefire.its.jiras;
-
-import org.apache.maven.surefire.its.fixture.SurefireJUnit4IntegrationTestCase;
-import org.junit.jupiter.api.Test;
-
-import static org.hamcrest.Matchers.containsString;
-import static org.hamcrest.Matchers.is;
-
-/**
- * A TestNG {@code ISuiteListener} registered through {@code
META-INF/services/org.testng.ITestNGListener}
- * should not be called more than needed.
- * <p>
- * The TestNG engine finds tests with a TestNG dry run, and a dry run calls
suite listeners just like a real
- * run. Surefire used to discover each scanned class on its own, so the
listener was called once per test
- * class, plus once for the discovery before the run, plus once for the run
itself. The scan no longer uses
- * the TestNG engine, so we are down to {@value #EXPECTED_SUITE_EVENTS}:
- * <ol>
- * <li>the discovery done right before the run,</li>
- * <li>the run of {@code TestNGTest}.</li>
- * </ol>
- * Getting down to one would need a change in the TestNG engine.
- *
- * @see <a href="https://github.com/apache/maven-surefire/issues/3462">GitHub
issue #3462</a>
- */
-public class Surefire3462TestNgSuiteListenerIT extends
SurefireJUnit4IntegrationTestCase {
-
- private static final int EXPECTED_SUITE_EVENTS = 2;
-
- @Test
- void suiteListenerIsNotInvokedOncePerScannedClass() throws Exception {
- unpack("surefire-3462-testng-suite-listener")
- .executeTest()
- .verifyErrorFree(2)
-
.assertThatLogLine(containsString("SUREFIRE-3462-SUITE-START"),
is(EXPECTED_SUITE_EVENTS))
-
.assertThatLogLine(containsString("SUREFIRE-3462-SUITE-FINISH"),
is(EXPECTED_SUITE_EVENTS));
- }
-
- /**
- * In a TestNG only project the TestNG engine is the only engine, so
skipping it during the scan must
- * not make JUnit warn that all engines were excluded.
- */
- @Test
- void testNgOnlyProjectScansWithoutWarning() throws Exception {
- unpack("surefire-3462-testng-only")
- .executeTest()
- .verifyErrorFree(1)
- .verifyTextNotInLog("All TestEngines were excluded")
-
.assertThatLogLine(containsString("SUREFIRE-3462-SUITE-START"),
is(EXPECTED_SUITE_EVENTS))
-
.assertThatLogLine(containsString("SUREFIRE-3462-SUITE-FINISH"),
is(EXPECTED_SUITE_EVENTS));
- }
-}
diff --git
a/surefire-its/src/test/resources/surefire-3462-testng-suite-listener/src/test/java/issue3462/TestNGTest.java
b/surefire-its/src/test/java/org/apache/maven/surefire/its/jiras/Surefire3548NestedParameterizedClassIT.java
similarity index 50%
rename from
surefire-its/src/test/resources/surefire-3462-testng-suite-listener/src/test/java/issue3462/TestNGTest.java
rename to
surefire-its/src/test/java/org/apache/maven/surefire/its/jiras/Surefire3548NestedParameterizedClassIT.java
index 61c9b3368..c58206ae9 100644
---
a/surefire-its/src/test/resources/surefire-3462-testng-suite-listener/src/test/java/issue3462/TestNGTest.java
+++
b/surefire-its/src/test/java/org/apache/maven/surefire/its/jiras/Surefire3548NestedParameterizedClassIT.java
@@ -16,16 +16,24 @@
* specific language governing permissions and limitations
* under the License.
*/
-package issue3462;
+package org.apache.maven.surefire.its.jiras;
-import org.testng.annotations.Test;
+import org.apache.maven.surefire.its.fixture.SurefireJUnit4IntegrationTestCase;
+import org.junit.jupiter.api.Test;
-import static org.testng.Assert.assertTrue;
-
-public class TestNGTest {
+/**
+ * A test in a nested {@code @ParameterizedClass} only exists once the outer
class runs. Selecting it
+ * with {@code -Dtest} must still run it, two outer values times two inner
values.
+ *
+ * @see <a href="https://github.com/apache/maven-surefire/issues/3548">GitHub
issue #3548</a>
+ */
+public class Surefire3548NestedParameterizedClassIT extends
SurefireJUnit4IntegrationTestCase {
@Test
- public void sanityCheck() {
- assertTrue(true);
+ void selectedTestInNestedParameterizedClassRuns() {
+ unpack("surefire-3548-nested-parameterized-class")
+
.setTestToRun("CalculatorParameterizedClassTests$Inner#regularTest")
+ .executeTest()
+ .verifyErrorFree(4);
}
}
diff --git a/surefire-its/src/test/resources/surefire-3462-testng-only/pom.xml
b/surefire-its/src/test/resources/surefire-3462-testng-only/pom.xml
deleted file mode 100644
index 2a3a13a34..000000000
--- a/surefire-its/src/test/resources/surefire-3462-testng-only/pom.xml
+++ /dev/null
@@ -1,66 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one
- ~ or more contributor license agreements. See the NOTICE file
- ~ distributed with this work for additional information
- ~ regarding copyright ownership. The ASF licenses this file
- ~ to you under the Apache License, Version 2.0 (the
- ~ "License"); you may not use this file except in compliance
- ~ with the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing,
- ~ software distributed under the License is distributed on an
- ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- ~ KIND, either express or implied. See the License for the
- ~ specific language governing permissions and limitations
- ~ under the License.
- -->
-
-<project xmlns="http://maven.apache.org/POM/4.0.0"
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
- <modelVersion>4.0.0</modelVersion>
-
- <groupId>org.apache.maven.plugins.surefire</groupId>
- <artifactId>surefire-3462-testng-only</artifactId>
- <version>1.0</version>
- <name>Test for: TestNG ISuiteListener in a TestNG only project</name>
-
- <properties>
- <maven.compiler.source>1.8</maven.compiler.source>
- <maven.compiler.target>1.8</maven.compiler.target>
- <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
- </properties>
-
- <dependencies>
- <!-- No testng-engine here on purpose: Surefire adds it when TestNG is
on the test class path,
- and that is what we test. -->
- <dependency>
- <groupId>org.testng</groupId>
- <artifactId>testng</artifactId>
- <version>7.5.1</version>
- <scope>test</scope>
- </dependency>
- </dependencies>
-
- <build>
- <plugins>
- <plugin>
- <artifactId>maven-compiler-plugin</artifactId>
- <version>3.8.0</version>
- <configuration>
- <encoding>UTF-8</encoding>
- </configuration>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-surefire-plugin</artifactId>
- <version>${surefire.version}</version>
- </plugin>
- </plugins>
- </build>
-
-</project>
diff --git
a/surefire-its/src/test/resources/surefire-3462-testng-only/src/test/java/issue3462/BaseTest.java
b/surefire-its/src/test/resources/surefire-3462-testng-only/src/test/java/issue3462/BaseTest.java
deleted file mode 100644
index 53825477c..000000000
---
a/surefire-its/src/test/resources/surefire-3462-testng-only/src/test/java/issue3462/BaseTest.java
+++ /dev/null
@@ -1,30 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-package issue3462;
-
-import org.testng.annotations.BeforeMethod;
-
-/**
- * Matches the scanner's name pattern, but TestNG skips abstract classes.
- */
-public abstract class BaseTest {
-
- @BeforeMethod
- public void setUp() {}
-}
diff --git
a/surefire-its/src/test/resources/surefire-3462-testng-only/src/test/java/issue3462/TestNGTest.java
b/surefire-its/src/test/resources/surefire-3462-testng-only/src/test/java/issue3462/TestNGTest.java
deleted file mode 100644
index 305422a55..000000000
---
a/surefire-its/src/test/resources/surefire-3462-testng-only/src/test/java/issue3462/TestNGTest.java
+++ /dev/null
@@ -1,31 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-package issue3462;
-
-import org.testng.annotations.Test;
-
-import static org.testng.Assert.assertTrue;
-
-public class TestNGTest extends BaseTest {
-
- @Test
- public void sanityCheck() {
- assertTrue(true);
- }
-}
diff --git
a/surefire-its/src/test/resources/surefire-3462-testng-only/src/test/java/issue3462/TrackingSuiteListener.java
b/surefire-its/src/test/resources/surefire-3462-testng-only/src/test/java/issue3462/TrackingSuiteListener.java
deleted file mode 100644
index c6b12faa2..000000000
---
a/surefire-its/src/test/resources/surefire-3462-testng-only/src/test/java/issue3462/TrackingSuiteListener.java
+++ /dev/null
@@ -1,38 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-package issue3462;
-
-import org.testng.ISuite;
-import org.testng.ISuiteListener;
-
-/**
- * Registered through {@code META-INF/services/org.testng.ITestNGListener},
like in the issue's reproducer.
- */
-public class TrackingSuiteListener implements ISuiteListener {
-
- @Override
- public void onStart(ISuite suite) {
- System.out.println("SUREFIRE-3462-SUITE-START");
- }
-
- @Override
- public void onFinish(ISuite suite) {
- System.out.println("SUREFIRE-3462-SUITE-FINISH");
- }
-}
diff --git
a/surefire-its/src/test/resources/surefire-3462-testng-only/src/test/resources/META-INF/services/org.testng.ITestNGListener
b/surefire-its/src/test/resources/surefire-3462-testng-only/src/test/resources/META-INF/services/org.testng.ITestNGListener
deleted file mode 100644
index 560aa645f..000000000
---
a/surefire-its/src/test/resources/surefire-3462-testng-only/src/test/resources/META-INF/services/org.testng.ITestNGListener
+++ /dev/null
@@ -1 +0,0 @@
-issue3462.TrackingSuiteListener
diff --git
a/surefire-its/src/test/resources/surefire-3462-testng-suite-listener/pom.xml
b/surefire-its/src/test/resources/surefire-3462-testng-suite-listener/pom.xml
deleted file mode 100644
index fedb86a85..000000000
---
a/surefire-its/src/test/resources/surefire-3462-testng-suite-listener/pom.xml
+++ /dev/null
@@ -1,72 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one
- ~ or more contributor license agreements. See the NOTICE file
- ~ distributed with this work for additional information
- ~ regarding copyright ownership. The ASF licenses this file
- ~ to you under the Apache License, Version 2.0 (the
- ~ "License"); you may not use this file except in compliance
- ~ with the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing,
- ~ software distributed under the License is distributed on an
- ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- ~ KIND, either express or implied. See the License for the
- ~ specific language governing permissions and limitations
- ~ under the License.
- -->
-
-<project xmlns="http://maven.apache.org/POM/4.0.0"
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
- <modelVersion>4.0.0</modelVersion>
-
- <groupId>org.apache.maven.plugins.surefire</groupId>
- <artifactId>surefire-3462-testng-suite-listener</artifactId>
- <version>1.0</version>
- <name>Test for: TestNG ISuiteListener is not invoked more often than
necessary</name>
-
- <properties>
- <maven.compiler.source>1.8</maven.compiler.source>
- <maven.compiler.target>1.8</maven.compiler.target>
- <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
- </properties>
-
- <dependencies>
- <dependency>
- <groupId>org.junit.jupiter</groupId>
- <artifactId>junit-jupiter-engine</artifactId>
- <version>5.11.4</version>
- <scope>test</scope>
- </dependency>
- <!-- The testng-engine is deliberately not declared here: Surefire
adds it automatically
- as soon as TestNG is on the test class path, which is the code
path under test. -->
- <dependency>
- <groupId>org.testng</groupId>
- <artifactId>testng</artifactId>
- <version>7.5.1</version>
- <scope>test</scope>
- </dependency>
- </dependencies>
-
- <build>
- <plugins>
- <plugin>
- <artifactId>maven-compiler-plugin</artifactId>
- <version>3.8.0</version>
- <configuration>
- <encoding>UTF-8</encoding>
- </configuration>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-surefire-plugin</artifactId>
- <version>${surefire.version}</version>
- </plugin>
- </plugins>
- </build>
-
-</project>
diff --git
a/surefire-its/src/test/resources/surefire-3462-testng-suite-listener/src/test/java/issue3462/TrackingSuiteListener.java
b/surefire-its/src/test/resources/surefire-3462-testng-suite-listener/src/test/java/issue3462/TrackingSuiteListener.java
deleted file mode 100644
index 73546c9ce..000000000
---
a/surefire-its/src/test/resources/surefire-3462-testng-suite-listener/src/test/java/issue3462/TrackingSuiteListener.java
+++ /dev/null
@@ -1,39 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-package issue3462;
-
-import org.testng.ISuite;
-import org.testng.ISuiteListener;
-
-/**
- * Registered through {@code META-INF/services/org.testng.ITestNGListener},
exactly like in the
- * reproducer attached to the issue.
- */
-public class TrackingSuiteListener implements ISuiteListener {
-
- @Override
- public void onStart(ISuite suite) {
- System.out.println("SUREFIRE-3462-SUITE-START");
- }
-
- @Override
- public void onFinish(ISuite suite) {
- System.out.println("SUREFIRE-3462-SUITE-FINISH");
- }
-}
diff --git
a/surefire-its/src/test/resources/surefire-3462-testng-suite-listener/src/test/resources/META-INF/services/org.testng.ITestNGListener
b/surefire-its/src/test/resources/surefire-3462-testng-suite-listener/src/test/resources/META-INF/services/org.testng.ITestNGListener
deleted file mode 100644
index 560aa645f..000000000
---
a/surefire-its/src/test/resources/surefire-3462-testng-suite-listener/src/test/resources/META-INF/services/org.testng.ITestNGListener
+++ /dev/null
@@ -1 +0,0 @@
-issue3462.TrackingSuiteListener
diff --git
a/surefire-its/src/test/resources/surefire-3548-nested-parameterized-class/pom.xml
b/surefire-its/src/test/resources/surefire-3548-nested-parameterized-class/pom.xml
new file mode 100644
index 000000000..b09172c05
--- /dev/null
+++
b/surefire-its/src/test/resources/surefire-3548-nested-parameterized-class/pom.xml
@@ -0,0 +1,60 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one
+ ~ or more contributor license agreements. See the NOTICE file
+ ~ distributed with this work for additional information
+ ~ regarding copyright ownership. The ASF licenses this file
+ ~ to you under the Apache License, Version 2.0 (the
+ ~ "License"); you may not use this file except in compliance
+ ~ with the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing,
+ ~ software distributed under the License is distributed on an
+ ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ ~ KIND, either express or implied. See the License for the
+ ~ specific language governing permissions and limitations
+ ~ under the License.
+ -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <groupId>org.apache.maven.plugins.surefire</groupId>
+ <artifactId>surefire-3548-nested-parameterized-class</artifactId>
+ <version>1.0</version>
+ <name>Select a test in a nested @ParameterizedClass</name>
+
+ <properties>
+ <maven.compiler.source>1.8</maven.compiler.source>
+ <maven.compiler.target>1.8</maven.compiler.target>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+ <junit5.version>5.14.4</junit5.version>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.junit.jupiter</groupId>
+ <artifactId>junit-jupiter</artifactId>
+ <version>${junit5.version}</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <version>3.8.0</version>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <version>${surefire.version}</version>
+ </plugin>
+ </plugins>
+ </build>
+</project>
diff --git
a/surefire-its/src/test/resources/surefire-3462-testng-suite-listener/src/test/java/issue3462/JupiterTest.java
b/surefire-its/src/test/resources/surefire-3548-nested-parameterized-class/src/test/java/issue3548/CalculatorParameterizedClassTests.java
similarity index 55%
rename from
surefire-its/src/test/resources/surefire-3462-testng-suite-listener/src/test/java/issue3462/JupiterTest.java
rename to
surefire-its/src/test/resources/surefire-3548-nested-parameterized-class/src/test/java/issue3548/CalculatorParameterizedClassTests.java
index 560d6073c..77b5f7a60 100644
---
a/surefire-its/src/test/resources/surefire-3462-testng-suite-listener/src/test/java/issue3462/JupiterTest.java
+++
b/surefire-its/src/test/resources/surefire-3548-nested-parameterized-class/src/test/java/issue3548/CalculatorParameterizedClassTests.java
@@ -16,16 +16,38 @@
* specific language governing permissions and limitations
* under the License.
*/
-package issue3462;
+package issue3548;
+import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.Parameter;
+import org.junit.jupiter.params.ParameterizedClass;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
-import static org.junit.jupiter.api.Assertions.assertTrue;
+@ParameterizedClass
+@ValueSource(ints = {1, 2})
+class CalculatorParameterizedClassTests {
-class JupiterTest {
+ @Parameter
+ int i;
- @Test
- void sanityCheck() {
- assertTrue(true);
+ @ParameterizedTest
+ @ValueSource(ints = {1, 2})
+ void parameterizedTest(int j) {}
+
+ @Nested
+ @ParameterizedClass
+ @ValueSource(ints = {1, 2})
+ class Inner {
+
+ final int j;
+
+ Inner(int j) {
+ this.j = j;
+ }
+
+ @Test
+ void regularTest() {}
}
}
diff --git
a/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/JUnitPlatformProvider.java
b/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/JUnitPlatformProvider.java
index 7a4fc17e0..59a6fe2f7 100644
---
a/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/JUnitPlatformProvider.java
+++
b/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/JUnitPlatformProvider.java
@@ -24,8 +24,6 @@
import java.lang.annotation.Annotation;
import java.lang.reflect.AnnotatedElement;
import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.LinkedHashSet;
@@ -117,6 +115,8 @@ public class JUnitPlatformProvider extends AbstractProvider
{
private static final String VINTAGE_ENGINE_ID = "junit-vintage";
+ private static final String TESTNG_ENGINE_ID = "testng";
+
private final ProviderParameters parameters;
private final LauncherSessionFactory launcherSessionFactory;
@@ -127,8 +127,6 @@ public class JUnitPlatformProvider extends AbstractProvider
{
private Map<String, String> configurationParameters = new HashMap<>();
- private boolean excludeTestNgEngine;
-
// groups only TestNG can use, we warn about them when the tests start
private final List<String> testNgOnlyGroups = new ArrayList<>();
@@ -240,13 +238,9 @@ public RunResult invoke(Object forkTestSet) throws
TestSetFailedException, Repor
try (LauncherSessionAdapter launcherSession =
launcherSessionFactory.openSession(cancellationToken)) {
LauncherAdapter launcher = launcherSession.getLauncher();
if (forkTestSet instanceof TestsToRun) {
- // The TestNG engine is not skipped here on purpose: reading
the classes eagerly would
- // advance the iteration state which TestsToRun exposes
through iterated().
invokeAllTests(launcher, (TestsToRun) forkTestSet, adapter);
} else if (forkTestSet instanceof Class) {
- Class<?> testClass = (Class<?>) forkTestSet;
-
excludeTestNgEngineIfPossible(Collections.singletonList(testClass.getName()));
- invokeAllTests(launcher, fromClass(testClass), adapter);
+ invokeAllTests(launcher, fromClass((Class<?>) forkTestSet),
adapter);
} else if (forkTestSet == null) {
invokeAllTests(launcher, scanClasspath(launcher), adapter);
} else {
@@ -273,26 +267,12 @@ private static void setupJunitLogger() {
}
private TestsToRun scanClasspath(LauncherAdapter launcher) {
+ TestPlanScannerFilter filter = new TestPlanScannerFilter(launcher,
filters);
ScanResult scanResult = parameters.getScanResult();
- List<String> candidateClassNames = scannedClassNames(scanResult);
- excludeTestNgEngineIfPossible(candidateClassNames);
- Set<String> testNgClassNames =
-
TestNgTestClassDetector.findTestNgTestClasses(parameters.getTestClassLoader(),
candidateClassNames);
- TestPlanScannerFilter filter =
- new TestPlanScannerFilter(launcher, allFilters(),
candidateClassNames, testNgClassNames);
TestsToRun scannedClasses = scanResult.applyFilter(filter,
parameters.getTestClassLoader());
return
parameters.getRunOrderCalculator().orderTestClasses(scannedClasses);
}
- private static List<String> scannedClassNames(ScanResult scanResult) {
- int size = scanResult.size();
- List<String> classNames = new ArrayList<>(size);
- for (int i = 0; i < size; i++) {
- classNames.add(scanResult.getClassName(i));
- }
- return classNames;
- }
-
private void invokeAllTests(LauncherAdapter launcher, TestsToRun
testsToRun, RunListenerAdapter adapter)
throws TestSetFailedException {
@@ -458,32 +438,7 @@ LauncherDiscoveryRequest
buildLauncherDiscoveryRequestForRerunFailures(RunListen
private LauncherDiscoveryRequestBuilder newRequest() {
- return
request().filters(allFilters()).configurationParameters(getConfigurationParameters());
- }
-
- /**
- * @return the include/exclude filters, extended by an engine filter which
keeps the TestNG engine out of
- * the request when it was proven that it cannot contribute any test
- */
- private Filter<?>[] allFilters() {
- if (!excludeTestNgEngine) {
- return filters;
- }
- Filter<?>[] allFilters = Arrays.copyOf(filters, filters.length + 1);
- allFilters[filters.length] =
EngineFilter.excludeEngines(TestNgTestClassDetector.TESTNG_ENGINE_ID);
- return allFilters;
- }
-
- /**
- * Excludes the TestNG engine when none of the given classes could be a
TestNG test class. The engine must
- * never be skipped for a class which was not inspected, so this is only
ever called with the complete set
- * of classes which is about to be handed over to the platform.
- *
- * @param classNames the names of every class which is about to be handed
over to the platform
- */
- private void excludeTestNgEngineIfPossible(Collection<String> classNames) {
- excludeTestNgEngine =
-
TestNgTestClassDetector.canExcludeTestNgEngine(parameters.getTestClassLoader(),
classNames);
+ return
request().filters(filters).configurationParameters(getConfigurationParameters());
}
private Filter<?>[] newFilters() {
@@ -673,7 +628,7 @@ private static PostDiscoveryFilter
byEngine(PostDiscoveryFilter vintageFilter, P
* TestNG tests always pass here, TestNG checks their groups itself.
*/
private static PostDiscoveryFilter
leaveTestNgTestsAlone(PostDiscoveryFilter filter) {
- return testDescriptor -> isFromEngine(testDescriptor,
TestNgTestClassDetector.TESTNG_ENGINE_ID)
+ return testDescriptor -> isFromEngine(testDescriptor, TESTNG_ENGINE_ID)
? FilterResult.included("TestNG matches the groups itself")
: filter.apply(testDescriptor);
}
diff --git
a/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/TestNgTestClassDetector.java
b/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/TestNgTestClassDetector.java
deleted file mode 100644
index d3e68caba..000000000
---
a/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/TestNgTestClassDetector.java
+++ /dev/null
@@ -1,161 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-package org.apache.maven.surefire.junitplatform;
-
-import java.lang.annotation.Annotation;
-import java.lang.reflect.Method;
-import java.lang.reflect.Modifier;
-import java.util.Collection;
-import java.util.LinkedHashSet;
-import java.util.Set;
-
-import org.apache.maven.surefire.api.util.ReflectionUtils;
-
-/**
- * Tells whether a class could be a TestNG test class.
- * <p>
- * The TestNG engine starts a real TestNG suite whenever it is asked to
discover tests, because it uses
- * TestNG's dry-run mode to tell test classes from other classes. It does so
for every class selector it
- * receives, even for classes which are obviously not TestNG tests, and a
started suite notifies every
- * registered {@code ISuiteListener}. So Surefire leaves the engine out when
none of the classes could be a
- * TestNG test, and it never asks the engine while scanning: TestNG test
classes are spotted here instead.
- * <p>
- * The detection mirrors the engine's own {@code IsTestNGTestClass}: a class
qualifies when
- * {@code org.testng.annotations.Test} is present on the class hierarchy or
directly on one of its methods.
- *
- * @since 3.6.1
- */
-final class TestNgTestClassDetector {
-
- static final String TESTNG_ENGINE_ID = "testng";
-
- private static final String TESTNG_TEST_ANNOTATION =
"org.testng.annotations.Test";
-
- private final Class<? extends Annotation> testAnnotation;
-
- TestNgTestClassDetector(Class<? extends Annotation> testAnnotation) {
- this.testAnnotation = testAnnotation;
- }
-
- /**
- * @param testClassLoader the class loader of the tests, may be {@code
null}
- * @return a detector, or {@code null} when TestNG is not on the test
class path and the engine can
- * therefore not contribute anything anyway
- */
- static TestNgTestClassDetector tryCreate(ClassLoader testClassLoader) {
- if (testClassLoader == null) {
- return null;
- }
- Class<?> annotation = ReflectionUtils.tryLoadClass(testClassLoader,
TESTNG_TEST_ANNOTATION);
- if (annotation == null || !annotation.isAnnotation()) {
- return null;
- }
- return new
TestNgTestClassDetector(annotation.asSubclass(Annotation.class));
- }
-
- /**
- * @param testClassLoader the class loader of the tests
- * @param classNames the names of every class which is about to be handed
over to the platform
- * @return {@code true} when the TestNG engine can safely be excluded from
the discovery request
- */
- static boolean canExcludeTestNgEngine(ClassLoader testClassLoader,
Collection<String> classNames) {
- TestNgTestClassDetector detector = tryCreate(testClassLoader);
- if (detector == null) {
- return false;
- }
- for (String className : classNames) {
- Class<?> testClass = ReflectionUtils.tryLoadClass(testClassLoader,
className);
- if (testClass == null || detector.isTestNgTestClass(testClass)) {
- // Unknown classes are left to the engine, only certainty
allows to skip it.
- return false;
- }
- }
- return true;
- }
-
- /**
- * Finds the scanned classes TestNG would run.
- *
- * @param testClassLoader the class loader of the tests
- * @param classNames the scanned class names
- * @return the TestNG test classes, empty if TestNG is not on the test
class path
- */
- static Set<String> findTestNgTestClasses(ClassLoader testClassLoader,
Collection<String> classNames) {
- Set<String> testNgClassNames = new LinkedHashSet<>();
- TestNgTestClassDetector detector = tryCreate(testClassLoader);
- if (detector == null) {
- return testNgClassNames;
- }
- for (String className : classNames) {
- Class<?> testClass = ReflectionUtils.tryLoadClass(testClassLoader,
className);
- if (testClass != null &&
detector.isRunnableTestNgTestClass(testClass)) {
- testNgClassNames.add(className);
- }
- }
- return testNgClassNames;
- }
-
- /**
- * Like {@link #isTestNgTestClass(Class)}, but TestNG skips abstract
classes and interfaces, so we do too.
- */
- boolean isRunnableTestNgTestClass(Class<?> candidate) {
- return !Modifier.isAbstract(candidate.getModifiers()) &&
isTestNgTestClass(candidate);
- }
-
- boolean isTestNgTestClass(Class<?> candidate) {
- try {
- return isAnnotatedInHierarchy(candidate) ||
hasMethodWithTestAnnotation(candidate);
- } catch (LinkageError e) {
- // The class cannot be inspected, so it cannot be ruled out either.
- return true;
- }
- }
-
- private boolean isAnnotatedInHierarchy(Class<?> candidate) {
- for (Class<?> clazz = candidate; clazz != null && clazz !=
Object.class; clazz = clazz.getSuperclass()) {
- if (clazz.getAnnotation(testAnnotation) != null) {
- return true;
- }
- }
- return false;
- }
-
- private boolean hasMethodWithTestAnnotation(Class<?> candidate) {
- for (Class<?> clazz = candidate; clazz != null && clazz !=
Object.class; clazz = clazz.getSuperclass()) {
- if (hasDeclaredMethodWithTestAnnotation(clazz)) {
- return true;
- }
- for (Class<?> implemented : clazz.getInterfaces()) {
- if (hasDeclaredMethodWithTestAnnotation(implemented)) {
- return true;
- }
- }
- }
- return false;
- }
-
- private boolean hasDeclaredMethodWithTestAnnotation(Class<?> clazz) {
- for (Method method : clazz.getDeclaredMethods()) {
- if (method.getAnnotation(testAnnotation) != null) {
- return true;
- }
- }
- return false;
- }
-}
diff --git
a/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/TestPlanScannerFilter.java
b/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/TestPlanScannerFilter.java
index 9a6bd5a2c..8172235ca 100644
---
a/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/TestPlanScannerFilter.java
+++
b/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/TestPlanScannerFilter.java
@@ -18,155 +18,45 @@
*/
package org.apache.maven.surefire.junitplatform;
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.Collection;
-import java.util.HashSet;
-import java.util.LinkedHashSet;
-import java.util.List;
-import java.util.Optional;
-import java.util.ServiceConfigurationError;
-import java.util.ServiceLoader;
-import java.util.Set;
-
import org.apache.maven.surefire.api.util.ScannerFilter;
-import org.junit.platform.engine.DiscoverySelector;
import org.junit.platform.engine.Filter;
-import org.junit.platform.engine.TestEngine;
-import org.junit.platform.engine.TestSource;
-import org.junit.platform.engine.support.descriptor.ClassSource;
-import org.junit.platform.engine.support.descriptor.MethodSource;
-import org.junit.platform.launcher.EngineFilter;
import org.junit.platform.launcher.LauncherDiscoveryRequest;
-import org.junit.platform.launcher.TestIdentifier;
import org.junit.platform.launcher.TestPlan;
import static
org.junit.platform.engine.discovery.DiscoverySelectors.selectClass;
import static
org.junit.platform.launcher.core.LauncherDiscoveryRequestBuilder.request;
/**
- * Accepts the scanned classes the JUnit Platform can actually run tests for.
- * <p>
- * All candidate classes are discovered in a <em>single</em> discovery request
instead of one request per
- * class. Discovery is not free: engines may execute arbitrary user code while
discovering, and the TestNG
- * engine even starts a real TestNG suite because it relies on TestNG's
dry-run mode. Discovering once per
- * scanned class multiplied those side effects by the number of test classes.
+ * Accepts the scanned classes the JUnit Platform has tests for.
* <p>
- * TestNG test classes skip discovery: {@link TestNgTestClassDetector} spots
them and they are accepted as
- * is. The other classes are discovered without the TestNG engine, so the only
TestNG dry run left is the
- * one done right before the tests run.
+ * Each class is discovered on its own and we only ask {@link
TestPlan#containsTests()}, which also counts
+ * containers that add their tests while running. Discovering all classes at
once and matching the test plan
+ * back to class names looks cheaper, but engines don't have to report a
source named after the selected
+ * class: a nested {@code @ParameterizedClass}, for example, only shows up
under its outer class (#3548).
*
* @since 2.22.0
*/
final class TestPlanScannerFilter implements ScannerFilter {
- private final Set<String> classNamesWithTests;
+ private final LauncherAdapter launcher;
+
+ private final Filter<?>[] includeAndExcludeFilters;
- /**
- * @param launcher discovers the classes which are not TestNG test classes
- * @param includeAndExcludeFilters the filters for that discovery
- * @param candidateClassNames the scanned class names
- * @param testNgClassNames the TestNG test classes, accepted without
discovery
- */
- TestPlanScannerFilter(
- LauncherAdapter launcher,
- Filter<?>[] includeAndExcludeFilters,
- Collection<String> candidateClassNames,
- Collection<String> testNgClassNames) {
- Set<String> otherClassNames = new LinkedHashSet<>(candidateClassNames);
- otherClassNames.removeAll(testNgClassNames);
- Filter<?>[] filters = includeAndExcludeFilters;
- if (!testNgClassNames.isEmpty()) {
- if (!otherClassNames.isEmpty() && !hasEngineOtherThanTestNg()) {
- // Nothing but TestNG could run these, and they are not TestNG
tests.
- otherClassNames.clear();
- }
- filters = Arrays.copyOf(includeAndExcludeFilters,
includeAndExcludeFilters.length + 1);
- filters[includeAndExcludeFilters.length] =
-
EngineFilter.excludeEngines(TestNgTestClassDetector.TESTNG_ENGINE_ID);
- }
- classNamesWithTests = discoverClassNamesWithTests(launcher, filters,
otherClassNames);
- classNamesWithTests.addAll(testNgClassNames);
+ TestPlanScannerFilter(LauncherAdapter launcher, Filter<?>[]
includeAndExcludeFilters) {
+ this.launcher = launcher;
+ this.includeAndExcludeFilters = includeAndExcludeFilters;
}
@Override
@SuppressWarnings("rawtypes")
public boolean accept(Class testClass) {
- return classNamesWithTests.contains(testClass.getName());
- }
-
- private static Set<String> discoverClassNamesWithTests(
- LauncherAdapter launcher, Filter<?>[] includeAndExcludeFilters,
Collection<String> candidateClassNames) {
-
- Set<String> candidates = new LinkedHashSet<>(candidateClassNames);
- if (candidates.isEmpty()) {
- return new HashSet<>();
- }
-
- List<DiscoverySelector> selectors = new ArrayList<>(candidates.size());
- for (String className : candidates) {
- selectors.add(selectClass(className));
- }
-
- LauncherDiscoveryRequest discoveryRequest =
-
request().selectors(selectors).filters(includeAndExcludeFilters).build();
+ LauncherDiscoveryRequest discoveryRequest = request()
+ .selectors(selectClass(testClass.getName()))
+ .filters(includeAndExcludeFilters)
+ .build();
TestPlan testPlan = launcher.discover(discoveryRequest);
- // The launcher prunes every descriptor which neither contains nor may
register tests, so a class
- // which is still part of the test plan is a class the platform has
tests for.
- Set<String> discovered = new HashSet<>();
- for (TestIdentifier root : testPlan.getRoots()) {
- collectClassNames(testPlan, root, candidates, discovered);
- }
- return discovered;
- }
-
- /**
- * Finds the engines the same way the launcher does. If TestNG is the only
one, leaving it out would make
- * the launcher warn that all engines were excluded.
- */
- private static boolean hasEngineOtherThanTestNg() {
- ClassLoader classLoader =
Thread.currentThread().getContextClassLoader();
- if (classLoader == null) {
- classLoader = ClassLoader.getSystemClassLoader();
- }
- try {
- for (TestEngine engine : ServiceLoader.load(TestEngine.class,
classLoader)) {
- if
(!TestNgTestClassDetector.TESTNG_ENGINE_ID.equals(engine.getId())) {
- return true;
- }
- }
- return false;
- } catch (ServiceConfigurationError | LinkageError e) {
- // Leave broken engines to the launcher, like before.
- return true;
- }
- }
-
- private static void collectClassNames(
- TestPlan testPlan, TestIdentifier identifier, Set<String>
candidates, Set<String> collected) {
-
- Optional<String> className = className(identifier);
- if (className.isPresent() && candidates.contains(className.get())) {
- collected.add(className.get());
- }
- for (TestIdentifier child : testPlan.getChildren(identifier)) {
- collectClassNames(testPlan, child, candidates, collected);
- }
- }
-
- private static Optional<String> className(TestIdentifier identifier) {
- return identifier.getSource().map(TestPlanScannerFilter::className);
- }
-
- private static String className(TestSource source) {
- if (source instanceof ClassSource) {
- return ((ClassSource) source).getClassName();
- }
- if (source instanceof MethodSource) {
- return ((MethodSource) source).getClassName();
- }
- return null;
+ return testPlan.containsTests();
}
}
diff --git
a/surefire-providers/surefire-junit-platform/src/test/java/org/apache/maven/surefire/junitplatform/TestNgTestClassDetectorTest.java
b/surefire-providers/surefire-junit-platform/src/test/java/org/apache/maven/surefire/junitplatform/TestNgTestClassDetectorTest.java
deleted file mode 100644
index c234d6487..000000000
---
a/surefire-providers/surefire-junit-platform/src/test/java/org/apache/maven/surefire/junitplatform/TestNgTestClassDetectorTest.java
+++ /dev/null
@@ -1,150 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-package org.apache.maven.surefire.junitplatform;
-
-import java.lang.annotation.Retention;
-import java.lang.annotation.RetentionPolicy;
-
-import org.junit.jupiter.api.Test;
-
-import static java.util.Arrays.asList;
-import static java.util.Collections.emptyList;
-import static java.util.Collections.singletonList;
-import static org.junit.jupiter.api.Assertions.assertFalse;
-import static org.junit.jupiter.api.Assertions.assertTrue;
-
-/**
- * Unit tests for {@link TestNgTestClassDetector}.
- * <p>
- * TestNG itself is not a dependency of this module, so the detection logic is
exercised with a stand-in
- * annotation. Only the resolution of {@code org.testng.annotations.Test}
depends on the real TestNG.
- */
-class TestNgTestClassDetectorTest {
-
- private final TestNgTestClassDetector detector = new
TestNgTestClassDetector(TestNgLikeTest.class);
-
- @Test
- void classWithoutAnnotationIsNotATestNgTestClass() {
- assertFalse(detector.isTestNgTestClass(PlainClass.class));
- }
-
- @Test
- void classAnnotatedAtTypeLevelIsATestNgTestClass() {
- assertTrue(detector.isTestNgTestClass(AnnotatedClass.class));
- }
-
- @Test
- void classWithAnnotatedMethodIsATestNgTestClass() {
- assertTrue(detector.isTestNgTestClass(ClassWithAnnotatedMethod.class));
- }
-
- @Test
- void classInheritingTypeLevelAnnotationIsATestNgTestClass() {
- assertTrue(detector.isTestNgTestClass(SubclassOfAnnotatedClass.class));
- }
-
- @Test
- void classInheritingAnnotatedMethodIsATestNgTestClass() {
-
assertTrue(detector.isTestNgTestClass(SubclassOfClassWithAnnotatedMethod.class));
- }
-
- @Test
- void classImplementingInterfaceWithAnnotatedMethodIsATestNgTestClass() {
-
assertTrue(detector.isTestNgTestClass(ImplementsAnnotatedInterface.class));
- }
-
- @Test
- void concreteTestNgTestClassIsRunnable() {
-
assertTrue(detector.isRunnableTestNgTestClass(ClassWithAnnotatedMethod.class));
- }
-
- @Test
- void abstractTestNgTestClassIsNotRunnable() {
-
assertFalse(detector.isRunnableTestNgTestClass(AbstractClassWithAnnotatedMethod.class));
- }
-
- @Test
- void interfaceWithAnnotatedMethodIsNotRunnable() {
-
assertFalse(detector.isRunnableTestNgTestClass(AnnotatedInterface.class));
- }
-
- @Test
- void noTestNgTestClassIsFoundWhenTestNgIsNotOnTheClassPath() {
- assertTrue(TestNgTestClassDetector.findTestNgTestClasses(
- getClass().getClassLoader(),
singletonList(ClassWithAnnotatedMethod.class.getName()))
- .isEmpty());
- }
-
- @Test
- void engineIsNotExcludedWhenTestNgIsNotOnTheClassPath() {
- assertFalse(TestNgTestClassDetector.canExcludeTestNgEngine(
- getClass().getClassLoader(),
singletonList(PlainClass.class.getName())));
- }
-
- @Test
- void engineIsNotExcludedWithoutClassLoader() {
- assertFalse(TestNgTestClassDetector.canExcludeTestNgEngine(null,
emptyList()));
- }
-
- @Test
- void detectorIsNotCreatedWhenTestNgIsNotOnTheClassPath() {
-
assertFalse(TestNgTestClassDetector.tryCreate(getClass().getClassLoader()) !=
null);
- }
-
- @Test
- void engineIsNotExcludedWhenAClassCannotBeLoaded() {
- assertFalse(TestNgTestClassDetector.canExcludeTestNgEngine(
- getClass().getClassLoader(),
asList(PlainClass.class.getName(), "does.not.Exist")));
- }
-
- @Retention(RetentionPolicy.RUNTIME)
- @interface TestNgLikeTest {}
-
- static class PlainClass {
-
- void method() {}
- }
-
- @TestNgLikeTest
- static class AnnotatedClass {}
-
- static class SubclassOfAnnotatedClass extends AnnotatedClass {}
-
- static class ClassWithAnnotatedMethod {
-
- @TestNgLikeTest
- void test() {}
- }
-
- static class SubclassOfClassWithAnnotatedMethod extends
ClassWithAnnotatedMethod {}
-
- abstract static class AbstractClassWithAnnotatedMethod {
-
- @TestNgLikeTest
- void test() {}
- }
-
- interface AnnotatedInterface {
-
- @TestNgLikeTest
- void test();
- }
-
- abstract static class ImplementsAnnotatedInterface implements
AnnotatedInterface {}
-}
diff --git
a/surefire-providers/surefire-junit-platform/src/test/java/org/apache/maven/surefire/junitplatform/TestPlanScannerFilterTest.java
b/surefire-providers/surefire-junit-platform/src/test/java/org/apache/maven/surefire/junitplatform/TestPlanScannerFilterTest.java
index 3e6c1d3b8..4e0693d2c 100644
---
a/surefire-providers/surefire-junit-platform/src/test/java/org/apache/maven/surefire/junitplatform/TestPlanScannerFilterTest.java
+++
b/surefire-providers/surefire-junit-platform/src/test/java/org/apache/maven/surefire/junitplatform/TestPlanScannerFilterTest.java
@@ -18,8 +18,6 @@
*/
package org.apache.maven.surefire.junitplatform;
-import java.net.URL;
-import java.net.URLClassLoader;
import java.util.List;
import java.util.stream.Stream;
@@ -27,20 +25,13 @@
import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.TestFactory;
+import org.junit.jupiter.params.Parameter;
+import org.junit.jupiter.params.ParameterizedClass;
+import org.junit.jupiter.params.provider.ValueSource;
import org.junit.platform.engine.Filter;
-import org.junit.platform.launcher.EngineFilter;
-import org.junit.platform.launcher.Launcher;
-import org.junit.platform.launcher.LauncherDiscoveryListener;
-import org.junit.platform.launcher.LauncherDiscoveryRequest;
-import org.junit.platform.launcher.TestExecutionListener;
-import org.junit.platform.launcher.TestPlan;
import org.junit.platform.launcher.core.LauncherFactory;
-import static java.util.Arrays.asList;
import static java.util.Collections.emptyList;
-import static java.util.Collections.singletonList;
-import static java.util.stream.Collectors.toList;
-import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
@@ -53,197 +44,54 @@ public class TestPlanScannerFilterTest {
@org.junit.jupiter.api.Test
public void emptyClassIsNotAccepted() {
- assertFalse(newFilter(EmptyClass.class).accept(EmptyClass.class),
"does not accept empty class");
+ assertFalse(newFilter().accept(EmptyClass.class), "does not accept
empty class");
}
@org.junit.jupiter.api.Test
public void classWithNoTestMethodsIsNotAccepted() {
- assertFalse(
-
newFilter(ClassWithMethods.class).accept(ClassWithMethods.class),
- "does not accept class with no @Test methods");
+ assertFalse(newFilter().accept(ClassWithMethods.class), "does not
accept class with no @Test methods");
}
@org.junit.jupiter.api.Test
public void classWithTestMethodsIsAccepted() {
-
assertTrue(newFilter(ClassWithTestMethods.class).accept(ClassWithTestMethods.class));
+ assertTrue(newFilter().accept(ClassWithTestMethods.class));
}
@org.junit.jupiter.api.Test
public void classWithNestedTestClassIsAccepted() {
-
assertTrue(newFilter(ClassWithNestedTestClass.class).accept(ClassWithNestedTestClass.class));
+ assertTrue(newFilter().accept(ClassWithNestedTestClass.class));
}
@org.junit.jupiter.api.Test
public void classWithDeeplyNestedTestClassIsAccepted() {
-
assertTrue(newFilter(ClassWithDeeplyNestedTestClass.class).accept(ClassWithDeeplyNestedTestClass.class));
+ assertTrue(newFilter().accept(ClassWithDeeplyNestedTestClass.class));
}
@org.junit.jupiter.api.Test
public void classWithTestFactoryIsAccepted() {
-
assertTrue(newFilter(ClassWithTestFactory.class).accept(ClassWithTestFactory.class));
+ assertTrue(newFilter().accept(ClassWithTestFactory.class));
}
@org.junit.jupiter.api.Test
public void classWithNestedTestFactoryIsAccepted() {
-
assertTrue(newFilter(ClassWithNestedTestFactory.class).accept(ClassWithNestedTestFactory.class));
- }
-
- @org.junit.jupiter.api.Test
- public void allCandidatesAreDiscoveredInASingleRequest() {
- CountingLauncher launcher = new
CountingLauncher(LauncherFactory.create());
- TestPlanScannerFilter filter = new TestPlanScannerFilter(
- new LauncherAdapter(launcher, null),
- new Filter<?>[0],
- Stream.of(
- EmptyClass.class,
- ClassWithMethods.class,
- ClassWithTestMethods.class,
- ClassWithNestedTestClass.class,
- ClassWithTestFactory.class)
- .map(Class::getName)
- .collect(toList()),
- emptyList());
-
- assertEquals(1, launcher.discoveries, "discovers all candidate classes
at once");
- assertFalse(filter.accept(EmptyClass.class));
- assertFalse(filter.accept(ClassWithMethods.class));
- assertTrue(filter.accept(ClassWithTestMethods.class));
- assertTrue(filter.accept(ClassWithNestedTestClass.class));
- assertTrue(filter.accept(ClassWithTestFactory.class));
- }
-
- @org.junit.jupiter.api.Test
- public void nothingIsDiscoveredWithoutCandidates() {
- CountingLauncher launcher = new
CountingLauncher(LauncherFactory.create());
- TestPlanScannerFilter filter = new TestPlanScannerFilter(
- new LauncherAdapter(launcher, null), new Filter<?>[0],
emptyList(), emptyList());
-
- assertEquals(0, launcher.discoveries, "does not discover anything
without candidates");
- assertFalse(filter.accept(ClassWithTestMethods.class));
- }
-
- @org.junit.jupiter.api.Test
- public void classIsNotAcceptedWhenOnlyAnotherCandidateHasTests() {
- assertFalse(newFilter(EmptyClass.class,
ClassWithTestMethods.class).accept(EmptyClass.class));
- }
-
- @org.junit.jupiter.api.Test
- public void testNgClassesAreAcceptedWithoutAskingTheTestNgEngine() {
- CountingLauncher launcher = new
CountingLauncher(LauncherFactory.create());
- TestPlanScannerFilter filter = new TestPlanScannerFilter(
- new LauncherAdapter(launcher, null),
- new Filter<?>[0],
- asList(EmptyClass.class.getName(),
ClassWithTestMethods.class.getName(), TestNgClass.class.getName()),
- singletonList(TestNgClass.class.getName()));
-
- assertEquals(1, launcher.discoveries, "still discovers the other
classes");
- assertTrue(excludesTestNgEngine(launcher.lastRequest), "the TestNG
engine is left out of the discovery");
- assertTrue(filter.accept(TestNgClass.class));
- assertTrue(filter.accept(ClassWithTestMethods.class));
- assertFalse(filter.accept(EmptyClass.class));
- }
-
- @org.junit.jupiter.api.Test
- public void testNgEngineIsNotExcludedWithoutTestNgClasses() {
- CountingLauncher launcher = new
CountingLauncher(LauncherFactory.create());
- new TestPlanScannerFilter(
- new LauncherAdapter(launcher, null),
- new Filter<?>[0],
- singletonList(ClassWithTestMethods.class.getName()),
- emptyList());
-
- assertFalse(excludesTestNgEngine(launcher.lastRequest));
+ assertTrue(newFilter().accept(ClassWithNestedTestFactory.class));
}
+ /**
+ * The tests of a nested {@code @ParameterizedClass} only show up when the
outer class runs, so its
+ * discovery has nothing named after the nested class. It must still be
accepted (#3548).
+ */
@org.junit.jupiter.api.Test
- public void nothingIsDiscoveredWhenTestNgIsTheOnlyEngine() throws
Exception {
- CountingLauncher launcher = new
CountingLauncher(LauncherFactory.create());
- Thread thread = Thread.currentThread();
- ClassLoader previous = thread.getContextClassLoader();
- // This class loader sees no engine at all, which looks the same as
TestNG being the only one.
- try (URLClassLoader noEngines = new URLClassLoader(new URL[0], null)) {
- thread.setContextClassLoader(noEngines);
- TestPlanScannerFilter filter = new TestPlanScannerFilter(
- new LauncherAdapter(launcher, null),
- new Filter<?>[0],
- asList(EmptyClass.class.getName(),
TestNgClass.class.getName()),
- singletonList(TestNgClass.class.getName()));
-
- assertEquals(0, launcher.discoveries, "no engine is left to ask");
- assertTrue(filter.accept(TestNgClass.class));
- assertFalse(filter.accept(EmptyClass.class));
- } finally {
- thread.setContextClassLoader(previous);
- }
+ public void nestedParameterizedClassIsAccepted() {
+ assertTrue(newFilter().accept(ParameterizedOuter.Inner.class));
}
- private static boolean excludesTestNgEngine(LauncherDiscoveryRequest
request) {
- for (EngineFilter engineFilter : request.getEngineFilters()) {
- if (!engineFilter.isIncludeFilter()
- &&
engineFilter.getEngineIds().contains(TestNgTestClassDetector.TESTNG_ENGINE_ID))
{
- return true;
- }
- }
- return false;
- }
-
- private static TestPlanScannerFilter newFilter(Class<?>... candidates) {
- List<String> candidateClassNames =
- Stream.of(candidates).map(Class::getName).collect(toList());
- return new TestPlanScannerFilter(
- new LauncherAdapter(LauncherFactory.create(), null),
- new Filter<?>[0],
- candidateClassNames,
- emptyList());
- }
-
- private static final class CountingLauncher implements Launcher {
-
- private final Launcher delegate;
-
- private int discoveries;
-
- private LauncherDiscoveryRequest lastRequest;
-
- private CountingLauncher(Launcher delegate) {
- this.delegate = delegate;
- }
-
- @Override
- public void
registerLauncherDiscoveryListeners(LauncherDiscoveryListener... listeners) {
- delegate.registerLauncherDiscoveryListeners(listeners);
- }
-
- @Override
- public void registerTestExecutionListeners(TestExecutionListener...
listeners) {
- delegate.registerTestExecutionListeners(listeners);
- }
-
- @Override
- public TestPlan discover(LauncherDiscoveryRequest
launcherDiscoveryRequest) {
- discoveries++;
- lastRequest = launcherDiscoveryRequest;
- return delegate.discover(launcherDiscoveryRequest);
- }
-
- @Override
- public void execute(LauncherDiscoveryRequest launcherDiscoveryRequest,
TestExecutionListener... listeners) {
- delegate.execute(launcherDiscoveryRequest, listeners);
- }
-
- @Override
- public void execute(TestPlan testPlan, TestExecutionListener...
listeners) {
- delegate.execute(testPlan, listeners);
- }
+ private static TestPlanScannerFilter newFilter() {
+ return new TestPlanScannerFilter(new
LauncherAdapter(LauncherFactory.create(), null), new Filter<?>[0]);
}
static class EmptyClass {}
- /**
- * Plays a TestNG test class. Being in the TestNG list is enough, no
annotation needed.
- */
- static class TestNgClass {}
-
static class ClassWithMethods {
void method1() {}
@@ -309,4 +157,27 @@ List<DynamicTest> tests() {
}
}
}
+
+ @ParameterizedClass
+ @ValueSource(ints = {1, 2})
+ static class ParameterizedOuter {
+
+ @Parameter
+ int i;
+
+ @Nested
+ @ParameterizedClass
+ @ValueSource(ints = {1, 2})
+ class Inner {
+
+ final int j;
+
+ Inner(int j) {
+ this.j = j;
+ }
+
+ @org.junit.jupiter.api.Test
+ void test() {}
+ }
+ }
}