Jesse Glick created SUREFIRE-1433:
-------------------------------------
Summary: rerunFailingTestsCount does not help with crashes
Key: SUREFIRE-1433
URL: https://issues.apache.org/jira/browse/SUREFIRE-1433
Project: Maven Surefire
Issue Type: Bug
Components: process forking
Affects Versions: 2.20.1
Environment: Linux, Java 8u131, Maven 3.5.0
Reporter: Jesse Glick
Attachments: demo.zip
I have a CI job running lots of very slow tests, and am using
{{-Dsurefire.rerunFailingTestsCount=2 -Dmaven.test.failure.ignore
-Dsurefire.timeout=7200 -DreuseForks=false -DforkCount=2}} to run Maven. It
will typically run for about three hours, collecting a lot of passing tests, a
few flaking tests with subsequent passes, and a few true failures. But then it
will abruptly halt with the infamous
{code:none}
Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on
project ...: ExecutionException The forked VM terminated without properly
saying goodbye. VM crash or System.exit called?
{code}
The timing is unpredictable but the build seems to usually fail this way;
sometimes I get lucky and it runs through all the tests, taking a number of
hours. Diagnosing the crash is not likely to be straightforward, and frankly I
do not care much—I just want Surefire to try again. Yet the
{{rerunFailingTestsCount}} flag does not accomplish this—it only retries normal
failures.
You can see this from running the attached project. While you might expect it
to go through 100 runs of the test, some failing politely and some crashing,
instead you see something like
{code:none}
...
--- maven-surefire-plugin:2.20.1:test (default-test) @ demo ---
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running demo.CrashingTest
failing
failing
failing
crashing
Results:
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
------------------------------------------------------------------------
BUILD FAILURE
------------------------------------------------------------------------
Total time: 2.456 s
Finished at: 2017-10-07T15:11:04-04:00
Final Memory: 11M/222M
------------------------------------------------------------------------
Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on
project demo: There are test failures.
...
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
