[jira] [Comment Edited] (SUREFIRE-1480) parallel tests may produce invalid .xml report
[
https://issues.apache.org/jira/browse/SUREFIRE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426273#comment-16426273
]
Mark Lehky edited comment on SUREFIRE-1480 at 4/4/18 10:36 PM:
---
I checked out the latest (3.0.0-SNAPSHOT), and from maven-surefire directory I
can {{mvn verify}} everything: "BUILD SUCCESS".
However, I cannot say the same from my IDE - I am an Eclipse guy. I get 138
errors. Any hints?
I can find the class you mentioned no problem. Does the surefire project use
any kind of a logger? For now I'm going to see how far I can get with just
{{System.out}}.
was (Author: mlehky):
I checked out the latest (3.0.0-SNAPSHOT), and from maven-surefire I can {{mvn
verify}} everything: "BUILD SUCCESS".
However, I cannot say the same from my IDE - I am an Eclipse guy. I get 138
errors. Any hints?
I can find the class you mentioned no problem. Does the surefire project use
any kind of a logger? For now I'm going to see how far I can get with just
{{System.out}}.
> parallel tests may produce invalid .xml report
> --
>
> Key: SUREFIRE-1480
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1480
> Project: Maven Surefire
> Issue Type: Bug
> Components: Junit 4.7+ (parallel) support, Maven Failsafe Plugin,
> Maven Surefire Plugin
>Affects Versions: 2.20.1
>Reporter: Mark Lehky
>Priority: Major
> Attachments: FailedXMLReport.txt, Stacktrace_failedTest.txt
>
>
> Relevant software:
> * Jenkins 2.108
> * Maven 3.??
> * JUnit 4.12
> * maven-failsafe-plugin 2.20.1 (I have seen this issue with surefire as well)
> I have a testsuite (one JUnit class) that contains multiple tests (multiple
> JUnit methods), which are all run in parallel. Some of the tests may be
> ignore using JUnit {{Assume}}.
> On occasion (not 100% reproducible), the resulting report will contain an
> entry like:
> {noformat}
> < message="Skip test!">
> {noformat}
> The correct entry, as is produced most of the time, should be:
> {noformat}
>
> {noformat}
> The invalid formatted XML, when run in Jenkins, results in the test being
> flagged as failed, and Jenkins simply has the message:
> "TEST-xml.[failed-to-read]" (the dots are replaced with the correct
> filename!).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (SUREFIRE-1480) parallel tests may produce invalid .xml report
[
https://issues.apache.org/jira/browse/SUREFIRE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424217#comment-16424217
]
Mark Lehky edited comment on SUREFIRE-1480 at 4/3/18 4:41 PM:
--
Right now I have a testsuite of ~190 tests, that runs for ~5 hours, using 2
parallel threads, and reproduces this problem ~50% of the time. Yesterday I was
able to confirm that Jenkins is not a dependency - I can reproduce this problem
just running my testsuite from the CLI: {{mvn verify}}. The (perhaps)
interesting thing is that out of 13 test classes, only one ends up being
mangled each time. Different one each time, but only one.
I am still trying to create something smaller than 5 hours to reproduce this.
was (Author: mlehky):
Right now I have a testsuite of ~190 tests, that runs for ~5 hours, and
reproduces this problem ~50% of the time. Yesterday I was able to confirm that
Jenkins is not a dependency - I can reproduce this problem just running my
testsuite from the CLI: {{mvn verify}}. The (perhaps) interesting thing is that
out of 13 test classes, only one ends up being mangled each time. Different one
each time, but only one.
I am still trying to create something smaller than 5 hours to reproduce this.
> parallel tests may produce invalid .xml report
> --
>
> Key: SUREFIRE-1480
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1480
> Project: Maven Surefire
> Issue Type: Bug
> Components: Junit 4.7+ (parallel) support, Maven Failsafe Plugin,
> Maven Surefire Plugin
>Affects Versions: 2.20.1
>Reporter: Mark Lehky
>Priority: Major
> Attachments: FailedXMLReport.txt, Stacktrace_failedTest.txt
>
>
> Relevant software:
> * Jenkins 2.108
> * Maven 3.??
> * JUnit 4.12
> * maven-failsafe-plugin 2.20.1 (I have seen this issue with surefire as well)
> I have a testsuite (one JUnit class) that contains multiple tests (multiple
> JUnit methods), which are all run in parallel. Some of the tests may be
> ignore using JUnit {{Assume}}.
> On occasion (not 100% reproducible), the resulting report will contain an
> entry like:
> {noformat}
> < message="Skip test!">
> {noformat}
> The correct entry, as is produced most of the time, should be:
> {noformat}
>
> {noformat}
> The invalid formatted XML, when run in Jenkins, results in the test being
> flagged as failed, and Jenkins simply has the message:
> "TEST-xml.[failed-to-read]" (the dots are replaced with the correct
> filename!).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
