Re: Maven surefire plugin: parallel configuration not running tests in parallel
My plan is to run these serially for local development and in parallel for
TeamCity, and for TeamCity I think it will pick out the logs using flowId, so
hopefully that will work.
From: Tibor Digana
Date: Wednesday, February 17, 2021 at 2:19 AM
To: Maven Users List
Subject: Re: Maven surefire plugin: parallel configuration not running tests in
parallel
In case of combining JUnit5 and Surefire/Failsafe, the configuration
parameters e.g. "parallel" and "threadCountClasses" are not bound to the
native JUnit parameters "junit.jupiter.execution.parallel.enabled" because
it was an agreement between Maven/JUnit teams and the solution became "by
design". We can talk about a new concept and bind these parameters of
course.
Regarding the parallel execution has it's own logging problems in plugin on
the top of JUnit5 engine but that's another issue you may be facing. We are
approaching the fix step by step.
Cheers
Tibor
On Tue, Feb 16, 2021 at 11:04 PM Jay Crosley wrote:
> I’m trying to get junit5 tests to run in parallel using the maven surefire
> plugin, as described on
> https://urldefense.proofpoint.com/v2/url?u=https-3A__maven.apache.org_surefire_maven-2Dsurefire-2Dplugin_examples_fork-2Doptions-2Dand-2Dparallel-2Dexecution.html=DwIFaQ=0hefKdg9jtsMu47wpF0ovg=qyxGgg8iek4zUTwPHqQd2x5bP20ZI2bxMqb2S9cASmw=M_2Gjt2FtuqAHQQ0HhrtLwLB6txc8A381wmGzx0q2jw=9y-T5guZoywkTqejMKcpEcM60c0Eg_HNafGvBD39cxM=
> .
> Despite configuration that looks correct, I can’t get them to run in
> parallel. I’ll paste my configuration and what I’ve tried and experienced
> so far. Any help is greatly appreciated!
>
> My surefire plugin configuration looks like this:
>
>
>
> org.apache.maven.plugins
> maven-surefire-plugin
> 2.21.0
>
>
> org.junit.platform
> junit-platform-surefire-provider
> 1.2.0
>
>
> org.junit.jupiter
> junit-jupiter-engine
> 5.2.0
>
>
>
>
> And I have a maven profile setup with additional configuration for our
> integration tests, which includes the parallel configuration. The commented
> out configurations indicate all the things I’ve tried.
>
>
>
> integration-tests-local
>
>
>
> org.apache.maven.plugins
> maven-surefire-plugin
> ${surefire.version}
>
> classes
> 4
> true
>
>
>
>
>
> false
>
> false
>
> none
>
>
> **/*IntegrationTests*.java
>
>
>
>
>
>
> The project is structured with a parent pom.xml and several sub-projects.
> The tests are in the “integration-tests” module, which pulls in the
> surefire plugin with no additional configuration:
>
>
>
> maven-surefire-plugin
>
>
> I mention the sub-project because it means that the maven command I’m
> running looks like this (I’ve tried with/without the “-T”):
>
>
> mvn -T 4 test -e -pl integration-tests -am -Pintegration-tests-local
>
> For the purposes of debugging, I’ve created 10 .java files named
> TestIntegrationTests1.java through TestIntegrationTests10.java, each of
> which has 10 unit tests all of which look like this:
>
>
> package com.axon.scorpius.integration_tests;
>
> import static org.junit.jupiter.api.Assertions.assertTrue;
>
> import org.junit.jupiter.api.Test;
>
> /**
> * Tests.
> */
> public class TestIntegrationTests1 {
>
> @Test
> void test1() {
> try {
> Thread.sleep(1000);
> } catch (InterruptedException ex) {
> System.out.println("Interrupted exception: " + ex);
> }
>
> assertTrue(true);
> }
>
>
>
> … 9 identical tests
>
> My hope is that when I run “mvn test” (I’m running locally in iTerm on a
> Macbook), these 10 test classes will run in parallel (at least partially),
> but they run serially, as seen here:
>
> INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests4
> [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 11.057 s - in com.axon.scorpius.integration_tests.TestIntegrationTests4
> [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests6
> [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 11.029 s - in com.axon.scorpius.integration_tests.TestIntegrationTests6
> [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests10
> [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 11.025 s - in com.axon.scorpius.integration_tests.TestIntegrationTests10
> [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests2
> [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 11.027 s - in com.axon.scorpius.integration_tests.TestIntegrationTests2
> [INFO] Running
Re: Maven surefire plugin: parallel configuration not running tests in parallel
In case of combining JUnit5 and Surefire/Failsafe, the configuration
parameters e.g. "parallel" and "threadCountClasses" are not bound to the
native JUnit parameters "junit.jupiter.execution.parallel.enabled" because
it was an agreement between Maven/JUnit teams and the solution became "by
design". We can talk about a new concept and bind these parameters of
course.
Regarding the parallel execution has it's own logging problems in plugin on
the top of JUnit5 engine but that's another issue you may be facing. We are
approaching the fix step by step.
Cheers
Tibor
On Tue, Feb 16, 2021 at 11:04 PM Jay Crosley wrote:
> I’m trying to get junit5 tests to run in parallel using the maven surefire
> plugin, as described on
> https://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html.
> Despite configuration that looks correct, I can’t get them to run in
> parallel. I’ll paste my configuration and what I’ve tried and experienced
> so far. Any help is greatly appreciated!
>
> My surefire plugin configuration looks like this:
>
>
>
> org.apache.maven.plugins
> maven-surefire-plugin
> 2.21.0
>
>
> org.junit.platform
> junit-platform-surefire-provider
> 1.2.0
>
>
> org.junit.jupiter
> junit-jupiter-engine
> 5.2.0
>
>
>
>
> And I have a maven profile setup with additional configuration for our
> integration tests, which includes the parallel configuration. The commented
> out configurations indicate all the things I’ve tried.
>
>
>
> integration-tests-local
>
>
>
> org.apache.maven.plugins
> maven-surefire-plugin
> ${surefire.version}
>
> classes
> 4
> true
>
>
>
>
>
> false
>
> false
>
> none
>
>
> **/*IntegrationTests*.java
>
>
>
>
>
>
> The project is structured with a parent pom.xml and several sub-projects.
> The tests are in the “integration-tests” module, which pulls in the
> surefire plugin with no additional configuration:
>
>
>
> maven-surefire-plugin
>
>
> I mention the sub-project because it means that the maven command I’m
> running looks like this (I’ve tried with/without the “-T”):
>
>
> mvn -T 4 test -e -pl integration-tests -am -Pintegration-tests-local
>
> For the purposes of debugging, I’ve created 10 .java files named
> TestIntegrationTests1.java through TestIntegrationTests10.java, each of
> which has 10 unit tests all of which look like this:
>
>
> package com.axon.scorpius.integration_tests;
>
> import static org.junit.jupiter.api.Assertions.assertTrue;
>
> import org.junit.jupiter.api.Test;
>
> /**
> * Tests.
> */
> public class TestIntegrationTests1 {
>
> @Test
> void test1() {
> try {
> Thread.sleep(1000);
> } catch (InterruptedException ex) {
> System.out.println("Interrupted exception: " + ex);
> }
>
> assertTrue(true);
> }
>
>
>
> … 9 identical tests
>
> My hope is that when I run “mvn test” (I’m running locally in iTerm on a
> Macbook), these 10 test classes will run in parallel (at least partially),
> but they run serially, as seen here:
>
> INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests4
> [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 11.057 s - in com.axon.scorpius.integration_tests.TestIntegrationTests4
> [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests6
> [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 11.029 s - in com.axon.scorpius.integration_tests.TestIntegrationTests6
> [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests10
> [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 11.025 s - in com.axon.scorpius.integration_tests.TestIntegrationTests10
> [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests2
> [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 11.027 s - in com.axon.scorpius.integration_tests.TestIntegrationTests2
> [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests7
> [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 11.032 s - in com.axon.scorpius.integration_tests.TestIntegrationTests7
> [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests5
> [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 11.027 s - in com.axon.scorpius.integration_tests.TestIntegrationTests5
> [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests1
> [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 11.032 s - in
