Hey Chad, Thanks for your response!
Which tool are you using to generate the "JUnit XML"? I don't believe there > is actually a proper schema for JUnit XML, and that it is a de-facto > "standard" that emerged from Ant originally, and then incorporated into > Maven/Gradle etc in their own XML report outputs. I am aware of an attempt > here <https://github.com/windyroad/JUnit-Schema> to build a schema, but > not actually sure which attribute might be used for this, and whether this > is also part of some de-facto standard. > The "tool" we are using to generate the XML is JUnit5's LegacyXmlReportGeneratingListener <https://junit.org/junit5/docs/5.5.2/api/org/junit/platform/reporting/legacy/xml/LegacyXmlReportGeneratingListener.html> (we've implemented our own test runner in order to get JUnit-style testing on the running system). According to the JUnit Jupiter docs <https://junit.org/junit5/docs/current/user-guide/#junit-platform-reporting>, that class generates a report in an "XML format...compatible with the de facto standard for JUnit 4 based test reports that was made popular by the Ant build system". I do not know whether they are following a schema, but they aren't doing any programmatic validation <https://github.com/junit-team/junit5/blob/main/junit-platform-reporting/src/main/java/org/junit/platform/reporting/legacy/xml/XmlReportWriter.java#L114>. There's obviously some interest in defining an up-to-date reporting schema to support new features of JUnit5 (JUnit5 issue 373 on GitHub <https://github.com/junit-team/junit5/issues/373>) but to the best of my knowledge there has been no movement on that. Contributions are welcome though - perhaps you can raise a GitHub issue > <https://github.com/gocd/gocd>with examples of the JUnit XML, and/or > submit a PR alongside some of the research into extensions to the "schema" > and any standardisation? > We considered submitting a patch to unittests.xsl <https://github.com/gocd/gocd/blob/master/common/src/main/resources/com/thoughtworks/go/domain/unittests.xsl>. However, the behavior we want is not a reasonable default and I don't see a way to configure the process that creates the results page. I will open a GitHub issue just to surface the problem, though. For the time being we've just implemented an additional TestExecutionListener <https://junit.org/junit5/docs/current/api/org.junit.platform.launcher/org/junit/platform/launcher/TestExecutionListener.html> which outputs a more useful format to the console and advised the development team to ignore the "Tests" tab. Thanks again for your help, ~Kenneth On Tue, Mar 1, 2022 at 8:07 AM Chad Wilson <[email protected]> wrote: > Hi Kenneth > > Which tool are you using to generate the "JUnit XML"? I don't believe > there is actually a proper schema for JUnit XML, and that it is a de-facto > "standard" that emerged from Ant originally, and then incorporated into > Maven/Gradle etc in their own XML report outputs. I am aware of an attempt > here <https://github.com/windyroad/JUnit-Schema> to build a schema, but > not actually sure which attribute might be used for this, and whether this > is also part of some de-facto standard. > > My inference is that GoCD seemed to move away from a focus on processing > test results like this (there are just so many different approaches across > language and tool ecosystems to keep up with), which is maybe why there are > missing pieces like this, and also evidenced by the relatively ugly, > unformatted view GoCD currently displays for the Tests tab. > > Personally, I think there are better tools for generating HTML reports > available via standard build automation tools (such as Gradle has), which > can then be artifacted and a custom tab added (similar to this > <https://docs.gocd.org/current/configuration/dev_upload_test_report.html#uploading-a-flash-video-and-displaying-it-as-a-sub-tab>) > to display that report. If GoCD did something with the XML (like tracking > repeatedly failing tests over time, slow tests etc to give you more > insight) it'd have more utility, but just for reporting on an individual > build, I prefer to use reports generated by the build tool/plugin. This is > the approach used by the GoCD team to build GoCD - based on > Gradle-generated HTML reports. :-) > > Contributions are welcome though - perhaps you can raise a GitHub issue > <https://github.com/gocd/gocd>with examples of the JUnit XML, and/or > submit a PR alongside some of the research into extensions to the "schema" > and any standardisation? > > -Chad > > On Tue, Mar 1, 2022 at 2:35 AM Kenneth Wayman <[email protected]> wrote: > >> *TL;DR* When running JUnit Jupiter parameterized tests, we would like >> the display name, not the indexed method name (i.e. "testMethod[1]"), to >> appear in the results on the "Tests" tab. >> >> We are using GoCD to run API tests against our compiled product. These >> are implemented as a series of JUnit Jupiter Parameterized tests, each of >> which runs with several hundred inputs. To give more descriptive names than >> "testTheAPI[1...235]", we modify the DisplayName of each test method based >> on the input. This approach works fine in the Eclipse IDE and from the >> command line. However, the JUnit XML results that GoCD is processing do not >> populate the "name" attribute of the "test-case" element with this value, >> so GoCD is still displaying "testTheApi[1]" etc. on the "Tests" tab. The >> DisplayName *IS* included in the result XML, it is just *NOT* in the >> attribute the UnitTestReportGenerator >> <https://github.com/gocd/gocd/blob/master/config/config-api/src/main/java/com/thoughtworks/go/domain/UnitTestReportGenerator.java> >> expects. >> Short of either rewriting the reports from JUnit 5 or modifying and >> patching unittests.xsl >> <https://github.com/gocd/gocd/blob/master/common/src/main/resources/com/thoughtworks/go/domain/unittests.xsl>, >> is there any way around this? >> >> -- >> You received this message because you are subscribed to the Google Groups >> "go-cd" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/go-cd/9aed64b7-16d8-42d6-8775-c68fc338b4b5n%40googlegroups.com >> <https://groups.google.com/d/msgid/go-cd/9aed64b7-16d8-42d6-8775-c68fc338b4b5n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to a topic in the > Google Groups "go-cd" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/go-cd/PqvJrhNTeag/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/CAA1RwH9MGfCtx_uCo7wnoYTrO7aLCNwNpbTVnKXjWtiTCx2eGQ%40mail.gmail.com > <https://groups.google.com/d/msgid/go-cd/CAA1RwH9MGfCtx_uCo7wnoYTrO7aLCNwNpbTVnKXjWtiTCx2eGQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CA%2BuBpr6DC32RNO5k75yoNEb3bmiLun-OS5r9550sTPqDbZG1eQ%40mail.gmail.com.
