On 19/04/2013, at 3:34 PM, Szczepan Faber <szczepan.fa...@gradleware.com> wrote:
> The schema for this file used to be somewhere in the web (I've found it some > time ago when I needed it). However, it's probably more important to be > compatible with most consumers (e.g. all/most CI servers). The fact that there is a schema doesn't really change anything, because it appears that no one respects it. > I think we should be very careful changing the format of the junit xml > reports (e.g. test out integrations with other CIs, etc). > > I think it would be very useful if Gradle supported output per test. The only way to do that (that we've identified) is to change the XML file, so this statement appears in contention with your previous. Are you saying we should change it, but be careful in doing so? Or should not change it and need to find another way? > > Cheers! > > > On Fri, Apr 19, 2013 at 2:33 PM, Luke Daley <luke.da...@gradleware.com> wrote: > Hi all, > > In the “JUnit XML” file that we currently produce we do not associate output > (std, err) with individual test cases, but with the suite (in practice: > class) as a whole. This is causing problems in integrating with > https://wiki.jenkins-ci.org/display/JENKINS/JUnit+Attachments+Plugin. > > Given: > > package junit; > > import org.junit.Test; > import org.junit.runner.RunWith; > import org.junit.runners.JUnit4; > > @RunWith(JUnit4.class) > public class TheTest { > > @Test > public void t() { > System.out.println("foo"); > } > > } > > We generate XML like: > > <?xml version="1.1" encoding="UTF-8"?> > <testsuite name="junit.TheTest" tests="1" failures="0" errors="0" > timestamp="1970-01-01T00:00:00" hostname="hasdrubal.local" > time="1.366372749405E9"> > <properties/> > <testcase name="t" classname="junit.TheTest" time="0.001"/> > <system-out><![CDATA[foo > ]]></system-out> > <system-err><![CDATA[]]></system-err> > </testsuite> > > After digging through the Jenkins code for how it parses these files, I > discovered that it supports: > > <?xml version="1.1" encoding="UTF-8"?> > <testsuite name="junit.TheTest" tests="1" failures="0" errors="0" > timestamp="1970-01-01T00:00:00" hostname="hasdrubal.local" > time="1.366372749405E9"> > <properties/> > <testcase name="t" classname="junit.TheTest" time="0.001"> > <system-out><![CDATA[foo > ]]></system-out> > <system-err><![CDATA[]]></system-err> > </testcase> > </testsuite> > > That is, you can associate output with an individual testcase. > > I did some experiments with Maven to see what it does. Maven doesn't record > any stdio information in the XML file usually; it puts text files on the > side. However, I happened upon a case where it does. If you try and execute a > non public class you get: > > <?xml version="1.0" encoding="UTF-8" ?> > <testsuite failures="0" time="0.006" errors="2" skipped="0" tests="2" > name="junit.TheTest"> > > <testcase time="0.002" classname="junit.TheTest" name="initializationError"> > <error message="Test class should have exactly one public constructor" > type="java.lang.Exception">java.lang.Exception: Test class should have > exactly one public constructor > at > org.junit.runners.BlockJUnit4ClassRunner.validateOnlyOneConstructor(BlockJUnit4ClassRunner.java:131) > … > </error> > <system-out>@SLTests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time > elapsed: 0.033 sec > @SLRunning junit.TheTest > </system-out> > </testcase> > <testcase time="0.001" classname="junit.TheTest" name="initializationError"> > <error message="Class junit.TheTest should be public" > type="java.lang.Exception">java.lang.Exception: Class junit.TheTest should be > public > at > org.junit.runners.model.FrameworkMethod.validatePublicVoid(FrameworkMethod.java:87) > … > </error> > </testcase> > </testsuite> > > The <system-out> elements are nested under <testcase> there. So it seems > there is some precedent for this. > > > I have some incentive to at least get this working for a client in Jenkins. > Where "get it working" would be to have the <system-out> nested under > <testcase> so it can do the correct association. > > I have no idea who else supports this format though. It's hard to predict > given that there is no schema for this XML file. > > -- > Luke Daley > Principal Engineer, Gradleware > http://gradleware.com > > Join me at the Gradle Summit 2013, June 13th and 14th in Santa Clara, CA: > http://www.gradlesummit.com > > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > > > > -- > Szczepan Faber > Principal engineer@gradleware; Lead@mockito > Join me at the Gradle Summit 2013, June 13th and 14th in Santa Clara, CA: > http://www.gradlesummit.com -- Luke Daley Principal Engineer, Gradleware http://gradleware.com Join me at the Gradle Summit 2013, June 13th and 14th in Santa Clara, CA: http://www.gradlesummit.com --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
