[
https://issues.apache.org/jira/browse/SUREFIRE-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16985644#comment-16985644
]
Tibor Digana commented on SUREFIRE-1728:
----------------------------------------
You could reply sooner. I was waiting for you!
ok, now I understand that you want to express an error or failure message in
the XML or TXT report.
The problem is that the timeout happens too late, after some classes were
serialized with their reports and the last one is being serialized on the disk
after the JVM was killed due to the timeout.
But yes, you are right when you say that you have no notion that the timeout
happened in the report.
Notice that the {{</testsuite>}} in the XML has no message attribute!
Would this new attribute be the right solution for you?
My question for you. Is the last class reported in the XML and is there
failures or errors in the testsuite? (see on the top of XML)
Even if this had a solution, there is another problem with the XSD schema of
the report because we must not touch it. And the reason is that the frameworks
like the NUnit or Jenkins JUnit plugin would have broken XSD schema validation
after our change.
I think this would require a contribution from both sides, all Jenkins plugins
and the Surefire.
> maven.test.failure.ignore: differentiate between test failure and timeout
> -------------------------------------------------------------------------
>
> Key: SUREFIRE-1728
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1728
> Project: Maven Surefire
> Issue Type: Improvement
> Affects Versions: 2.22.2, 3.0.0-M3
> Environment: Maven 3.6.2
> Reporter: Falko Modler
> Assignee: Tibor Digana
> Priority: Major
> Fix For: 3.0.0-M5
>
>
> On a build server like Jenkins, people typically set
> {{-Dmaven.test.failure.ignore=true}} to get the maximum number of test
> results instead of failing the build after the first module with test
> failures.
> Unfortunately, timeouts are also ignored when this property is activated,
> leaving the Jenkins JUnit plugin no chance to detect that something went
> wrong (because a timeout is not reported in a txt or xml report).
> See also [JENKINS-46553|https://issues.jenkins-ci.org/browse/JENKINS-46553].
> The two cases should be differentiated.
> Due to backward compatibility reasons, I am not sure whether it would be wise
> to simply exclude timeout cases.
> One backward compatible solution might be to extend the value range of
> {{maven.test.failure.ignore}} from just {{true}} XOR {{false}} to something
> like:
> {{true}}/{{all}} XOR {{failure}} XOR {{false}}.
> The alternative would be to introduce yet another property...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
