![]() |
|
|
|
|
Change By:
|
Julien Carsique
(10/Sep/14 3:42 PM)
|
|
Description:
|
I understood from various emails, issues, discussions and source code that it is a wanted behavior to set a build UNSTABLE when (surefire or failsafe) tests fail. That is consistent with https://wiki.jenkins-ci.org/display/JENKINS/Terminology
However, the current behavior is error-prone and inconsistent with a command line or Freestyle job Maven execution.
I guess that if the UNSTABLE status was that much important, it would have been implemented on Freestyle jobs too. Especially since Freestyle are so often recommended rather than the useful Maven job.
h3. Problem
First main issue is the build continuation.
Second issue is the behavior difference between Maven and Freestyle jobs.
h4. Freestyle job
The result is the same as an execution from command line:
- Maven execution is interrupted
- Maven build is FAILURE
- Jenkins build is FAILURE
Reproducing the Maven job behavior is partially possible with {{-Dmaven.test.failure.ignore=true}} option and the use of 'Publish JUnit test result report' post build step (to change the build result from SUCCESS to UNSTABLE).
h4. Maven job
The result is similar to using the {{-Dmaven.test.failure.ignore=true}} option:
- *the Maven execution is not interrupted*
- Maven build is SUCCESS
- Jenkins build is UNSTABLE
Responsible code is there: https://github.com/jenkinsci/maven-plugin/blob/maven-plugin-2.6/src/main/java/hudson/maven/reporters/SurefireArchiver.java#L88
The main issue is the build continuation, which will for instance install the artifacts:
{noformat}
[INFO] --- maven-surefire-plugin:2.17:test (default-test) @ test-jenkins-failures ---
[ERROR] There are test failures.
[INFO] --- maven-failsafe-plugin:2.17:integration-test (default) @ test-jenkins-failures ---
[INFO] --- maven-failsafe-plugin:2.17:verify (default) @ test-jenkins-failures ---
[ERROR] There are test failures.
[INFO] --- maven-install-plugin:2.5.1:install (default-install) @ test-jenkins-failures ---
[INFO] Installing ...
[INFO] BUILD SUCCESS
Finished: UNSTABLE{noformat}
ḧ3
h3
. Solutions
h4. Add an option for ignoring or not the test failures during the build.
An option would highlight that behavior and allow to easily switch from a testFailureIgnore mode to a standard mode.
h4. Rather use {{-fae}} Maven 3 option, or improve current implementation not to use the testFailureIgnore property hack
{noformat}-fae,--fail-at-end Only fail the build afterwards; allow all non-impacted builds to continue{noformat}
That would not be the exact equivalent to the current implementation but would still allow to get a status on multiple modules and would only require to "catch" the Maven failure at the end of the build.
Having a final Maven build FAILED translated to Jenkins build UNSTABLE would avoid the unwanted execution of other plugins after surefire or failsafe detected the failure.
h4. Consistency with Freestyle job
For consistency purpose, the 'Publish JUnit test result report' post build step should be able to switch the Jenkins build result from FAILURE to UNSTABLE when the BUILD failure is only due to Maven tests failure.
|
|
|
|
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
|
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit
https://groups.google.com/d/optout.
