[
https://jira.codehaus.org/browse/SUREFIRE-611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kristian Rosenvold closed SUREFIRE-611.
---------------------------------------
Resolution: Fixed
Fix Version/s: 2.11
Assignee: Kristian Rosenvold
Exit code handling revised in 2.11 and fixes this issue
> surefire exit code should explicitly distinguish between crashed and failed
> unit test(s)
> ----------------------------------------------------------------------------------------
>
> Key: SUREFIRE-611
> URL: https://jira.codehaus.org/browse/SUREFIRE-611
> Project: Maven Surefire
> Issue Type: Improvement
> Components: Maven Surefire Plugin
> Reporter: Tycho Lamerigts
> Assignee: Kristian Rosenvold
> Fix For: 2.11
>
> Attachments: tc-junit-crash.tar.gz
>
>
> - If a unit test crashes, surefire skips all remaining tests, prints a
> message "[ERROR] There are test failures. ..." to the console, and does NOT
> print a test summary report.
> - If one or more unit tests fail, maven completes remaining tests, prints a
> message "[ERROR] There are test failures. ..." to the console, and DOES
> print a test summary report.
> The only distinction between crash and fail is the presence (or absence) of a
> summary report.
> This is a problem for some tools (like TeamCity), because they typically
> don't pick up on the fact that many testcases were not run at all in case of
> a crashed unit test. Instead in case of a crashed test they simply report,
> for example, 100 tests succeeded, 0 failed. They do not report "50 tests
> skipped due to a crash after test #100", and it is quite hard for those tools
> to recognize this situation. If surefire would report a crash more
> explicitly, other tools can act accordingly.
> I have attached a small maven project to reproduce the problem. By commenting
> and uncommenting the marked line in file ATest.java, you can reproduce both
> fail and crash scenarios. In the crash scenario, Maven will correctly report
> a failed build (unless you use the -Dmaven.test.failure.ignore flag, in which
> case it even won't do that), but it fails to indicate there was a crash which
> caused tests (in this case 3) to be skipped.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
