On 19/04/2013, at 1:33 PM, Luke Daley <luke.da...@gradleware.com> wrote:
> Hi all, > > In the “JUnit XML” file that we currently produce we do not associate output > (std, err) with individual test cases, but with the suite (in practice: > class) as a whole. This is causing problems in integrating with > https://wiki.jenkins-ci.org/display/JENKINS/JUnit+Attachments+Plugin. > > Given: > > package junit; > > import org.junit.Test; > import org.junit.runner.RunWith; > import org.junit.runners.JUnit4; > > @RunWith(JUnit4.class) > public class TheTest { > > @Test > public void t() { > System.out.println("foo"); > } > > } > > We generate XML like: > > <?xml version="1.1" encoding="UTF-8"?> > <testsuite name="junit.TheTest" tests="1" failures="0" errors="0" > timestamp="1970-01-01T00:00:00" hostname="hasdrubal.local" > time="1.366372749405E9"> > <properties/> > <testcase name="t" classname="junit.TheTest" time="0.001"/> > <system-out><![CDATA[foo > ]]></system-out> > <system-err><![CDATA[]]></system-err> > </testsuite> > > After digging through the Jenkins code for how it parses these files, I > discovered that it supports: > > <?xml version="1.1" encoding="UTF-8"?> > <testsuite name="junit.TheTest" tests="1" failures="0" errors="0" > timestamp="1970-01-01T00:00:00" hostname="hasdrubal.local" > time="1.366372749405E9"> > <properties/> > <testcase name="t" classname="junit.TheTest" time="0.001"> > <system-out><![CDATA[foo > ]]></system-out> > <system-err><![CDATA[]]></system-err> > </testcase> > </testsuite> > > That is, you can associate output with an individual testcase. > > I did some experiments with Maven to see what it does. Maven doesn't record > any stdio information in the XML file usually; it puts text files on the > side. However, I happened upon a case where it does. If you try and execute a > non public class you get: > > <?xml version="1.0" encoding="UTF-8" ?> > <testsuite failures="0" time="0.006" errors="2" skipped="0" tests="2" > name="junit.TheTest"> > > <testcase time="0.002" classname="junit.TheTest" name="initializationError"> > <error message="Test class should have exactly one public constructor" > type="java.lang.Exception">java.lang.Exception: Test class should have > exactly one public constructor > at > org.junit.runners.BlockJUnit4ClassRunner.validateOnlyOneConstructor(BlockJUnit4ClassRunner.java:131) > … > </error> > <system-out>@SLTests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time > elapsed: 0.033 sec > @SLRunning junit.TheTest > </system-out> > </testcase> > <testcase time="0.001" classname="junit.TheTest" name="initializationError"> > <error message="Class junit.TheTest should be public" > type="java.lang.Exception">java.lang.Exception: Class junit.TheTest should be > public > at > org.junit.runners.model.FrameworkMethod.validatePublicVoid(FrameworkMethod.java:87) > … > </error> > </testcase> > </testsuite> > > The <system-out> elements are nested under <testcase> there. So it seems > there is some precedent for this. > > > I have some incentive to at least get this working for a client in Jenkins. > Where "get it working" would be to have the <system-out> nested under > <testcase> so it can do the correct association. > > I have no idea who else supports this format though. It's hard to predict > given that there is no schema for this XML file. Checked with the Bamboo guys, they don't capture stdout from tests at all. If we did have a nested output element it wouldn't break their parsing. -- Luke Daley Principal Engineer, Gradleware http://gradleware.com Join me at the Gradle Summit 2013, June 13th and 14th in Santa Clara, CA: http://www.gradlesummit.com --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
