[
https://issues.apache.org/jira/browse/SUREFIRE-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Emond Papegaaij updated SUREFIRE-1935:
--------------------------------------
Description:
When running tests with forkCount > 1 on the JUnit 5 Platform, the
{{JUnitPlatformProvider}} keeps restarting the entire launcher, causing the
life cycle of the tests to change. When an {{Extension}} registers an expensive
resource in the root store, this resource is now created over and over again
for every test class. An example of such an {{Extension}} is the
{{ArquillianExtension}}. The recreation of this resource causes the tests to
create and destroy all containers associated with the test for every class,
rather than once per fork.
I've attached a very simple example project that contains 4 tests and a dummy
extension that registers an object in the root store. The output clearly
indicates the problem:
{code}
$ mvn surefire:test -DforkCount=1
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------------< nl.topicus:forkedtests >-----------------------
[INFO] Building forkedtests 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-cli) @ forkedtests ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running nl.topicus.test.Test2
Performing heavy calculation
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.021 s
- in nl.topicus.test.Test2
[INFO] Running nl.topicus.test.Test1
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.001 s
- in nl.topicus.test.Test1
[INFO] Running nl.topicus.test.Test3
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.002 s
- in nl.topicus.test.Test3
[INFO] Running nl.topicus.test.Test4
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.001 s
- in nl.topicus.test.Test4
Discarding heavy calculation
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.816 s
[INFO] Finished at: 2021-08-13T12:50:40+02:00
[INFO] ------------------------------------------------------------------------
{code}
Compared to running with {{forkCount=2}} (notice how the heavy calculation is
performed 4 times, where it should only run twice):
{code}
$ mvn surefire:test -DforkCount=2
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------------< nl.topicus:forkedtests >-----------------------
[INFO] Building forkedtests 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-cli) @ forkedtests ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running nl.topicus.test.Test2
[INFO] Running nl.topicus.test.Test1
Performing heavy calculation
Performing heavy calculation
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.025 s
- in nl.topicus.test.Test2
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.022 s
- in nl.topicus.test.Test1
Discarding heavy calculation
Discarding heavy calculation
[INFO] Running nl.topicus.test.Test3
[INFO] Running nl.topicus.test.Test4
Performing heavy calculation
Performing heavy calculation
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.002 s
- in nl.topicus.test.Test3
Discarding heavy calculation
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.002 s
- in nl.topicus.test.Test4
Discarding heavy calculation
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.893 s
[INFO] Finished at: 2021-08-13T12:50:48+02:00
[INFO] ------------------------------------------------------------------------
{code}
was:
When running tests with forkCount > 1 on the JUnit 5 Platform, the
{{JUnitPlatformProvider}} keeps restarting the entire launcher, causing the
life cycle of the tests to change. When an {{Extension}} registers an expensive
resource in the root store, this resource is now created over and over again
for every test class. An example of such an {{Extension}} is the
{{ArquillianExtension}}. The recreation of this resource causes the tests to
create and destroy all containers associated with the test for every class,
rather than once per fork.
I've attached a very simple example project that contains 4 tests and a dummy
extension that registers an object in the root store. The output clearly
indicates the problem:
{code}
$ mvn surefire:test -DforkCount=1
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------------< nl.topicus:forkedtests >-----------------------
[INFO] Building forkedtests 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-cli) @ forkedtests ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running nl.topicus.test.Test2
Performing heavy calculation
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.021 s
- in nl.topicus.test.Test2
[INFO] Running nl.topicus.test.Test1
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.001 s
- in nl.topicus.test.Test1
[INFO] Running nl.topicus.test.Test3
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.002 s
- in nl.topicus.test.Test3
[INFO] Running nl.topicus.test.Test4
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.001 s
- in nl.topicus.test.Test4
Discarding heavy calculation
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.816 s
[INFO] Finished at: 2021-08-13T12:50:40+02:00
[INFO] ------------------------------------------------------------------------
{code}
Compared to running with {{forkCount=2}}:
{code}
$ mvn surefire:test -DforkCount=2
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------------< nl.topicus:forkedtests >-----------------------
[INFO] Building forkedtests 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-cli) @ forkedtests ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running nl.topicus.test.Test2
[INFO] Running nl.topicus.test.Test1
Performing heavy calculation
Performing heavy calculation
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.025 s
- in nl.topicus.test.Test2
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.022 s
- in nl.topicus.test.Test1
Discarding heavy calculation
Discarding heavy calculation
[INFO] Running nl.topicus.test.Test3
[INFO] Running nl.topicus.test.Test4
Performing heavy calculation
Performing heavy calculation
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.002 s
- in nl.topicus.test.Test3
Discarding heavy calculation
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.002 s
- in nl.topicus.test.Test4
Discarding heavy calculation
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.893 s
[INFO] Finished at: 2021-08-13T12:50:48+02:00
[INFO] ------------------------------------------------------------------------
{code}
> forkCount > 1 disrupts the JupiterTestEngine life cycle
> --------------------------------------------------------
>
> Key: SUREFIRE-1935
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1935
> Project: Maven Surefire
> Issue Type: Bug
> Components: JUnit 5.x support, process forking
> Affects Versions: 3.0.0-M5
> Reporter: Emond Papegaaij
> Priority: Major
> Attachments: forkedtests.zip
>
>
> When running tests with forkCount > 1 on the JUnit 5 Platform, the
> {{JUnitPlatformProvider}} keeps restarting the entire launcher, causing the
> life cycle of the tests to change. When an {{Extension}} registers an
> expensive resource in the root store, this resource is now created over and
> over again for every test class. An example of such an {{Extension}} is the
> {{ArquillianExtension}}. The recreation of this resource causes the tests to
> create and destroy all containers associated with the test for every class,
> rather than once per fork.
> I've attached a very simple example project that contains 4 tests and a dummy
> extension that registers an object in the root store. The output clearly
> indicates the problem:
> {code}
> $ mvn surefire:test -DforkCount=1
> [INFO] Scanning for projects...
> [INFO]
> [INFO] -----------------------< nl.topicus:forkedtests
> >-----------------------
> [INFO] Building forkedtests 1.0-SNAPSHOT
> [INFO] --------------------------------[ jar
> ]---------------------------------
> [INFO]
> [INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-cli) @ forkedtests ---
> [INFO]
> [INFO] -------------------------------------------------------
> [INFO] T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running nl.topicus.test.Test2
> Performing heavy calculation
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.021
> s - in nl.topicus.test.Test2
> [INFO] Running nl.topicus.test.Test1
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.001
> s - in nl.topicus.test.Test1
> [INFO] Running nl.topicus.test.Test3
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.002
> s - in nl.topicus.test.Test3
> [INFO] Running nl.topicus.test.Test4
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.001
> s - in nl.topicus.test.Test4
> Discarding heavy calculation
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 8.816 s
> [INFO] Finished at: 2021-08-13T12:50:40+02:00
> [INFO]
> ------------------------------------------------------------------------
> {code}
> Compared to running with {{forkCount=2}} (notice how the heavy calculation is
> performed 4 times, where it should only run twice):
> {code}
> $ mvn surefire:test -DforkCount=2
> [INFO] Scanning for projects...
> [INFO]
> [INFO] -----------------------< nl.topicus:forkedtests
> >-----------------------
> [INFO] Building forkedtests 1.0-SNAPSHOT
> [INFO] --------------------------------[ jar
> ]---------------------------------
> [INFO]
> [INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-cli) @ forkedtests ---
> [INFO]
> [INFO] -------------------------------------------------------
> [INFO] T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running nl.topicus.test.Test2
> [INFO] Running nl.topicus.test.Test1
> Performing heavy calculation
> Performing heavy calculation
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.025
> s - in nl.topicus.test.Test2
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.022
> s - in nl.topicus.test.Test1
> Discarding heavy calculation
> Discarding heavy calculation
> [INFO] Running nl.topicus.test.Test3
> [INFO] Running nl.topicus.test.Test4
> Performing heavy calculation
> Performing heavy calculation
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.002
> s - in nl.topicus.test.Test3
> Discarding heavy calculation
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.002
> s - in nl.topicus.test.Test4
> Discarding heavy calculation
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 4.893 s
> [INFO] Finished at: 2021-08-13T12:50:48+02:00
> [INFO]
> ------------------------------------------------------------------------
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
