Obviously, the choice is a bit ideological and context dependent. I wasn't trying to be contentious.
I'll have a look at some of the options you mentioned. On Friday, September 7, 2012 5:10:56 PM UTC-4, Mark Waite wrote: > > If that's what you want, then that's a good thing to do. I prefer having > "Unstable" mean "code compiled, one or more tests failed" so I gained more > meaning from the "red" / "yellow" / "blue" states of the build. If you > gain more meaning by having two states, "red" and "blue" for your results, > that's OK too. > > I had not considered the case where you truly have no compilation phase at > all, since all the projects I use involve a mixture of compiled and dynamic > languages, with the dynamic languages typically not being invoked until > after the compilation phase is complete. > > The cases where I've needed to run MSTest (a few years ago now) also > required that I be able to process more than one MSTest output file at a > time, and that capability was not available at that time (added in version > 0.7). I see that the capability to process multiple files with wildcards > has been available for over a year, so I'm glad it meets your needs. Using > the MSTest plugin seems like a good way to do it. > > I think the idea of causing your MSTest invocation to return a non-zero > exit code is still a reasonable approach to give you the failure you're > seeking when a test fails. You might also consider using the MSTestRunner > plugin to launch MSTest ( > https://wiki.jenkins-ci.org/display/JENKINS/MSTestRunner+Plugin ) to see > if it has a way to mark the job as failed when a test fails. Yet another > alternative would be to investigate the xUnit plugin to see if it will give > you the flexibility you want. > > Mark Waite > > > > ------------------------------ > *From:* bearrito <j.barrett...@gmail.com <javascript:>> > *To:* jenkins...@googlegroups.com <javascript:> > *Cc:* Mark Waite <mark...@yahoo.com <javascript:>> > *Sent:* Friday, September 7, 2012 11:35 AM > *Subject:* Re: MSTest Plugin doesn't fail the job on failing test. > > Not marking a build as failed with failing unit tests doesn't make any > sense to me. We fail the build if a compile fails so why not with tests? > What if I was working in a dynamic/interpreted language and so couldn't > rely on compiler/type system to find what would be guaranteed run-time > errors? > > I'd prefer to use the MSTest plugin because the MSTest plugin performs the > conversion from the .trx test output to the junit format that jenkins > understands. > > -barrett > > On Thursday, September 6, 2012 1:51:00 PM UTC-4, Mark Waite wrote: > > You might consider the comments in this thread before you mark a job as > failed due to test failures: > > http://jenkins.361315.n4. nabble.com/plugin-A-build- > with-failed-tests-should-be- marked-as-Unstable-or-Failure- > td3585663.html<http://jenkins.361315.n4.nabble.com/plugin-A-build-with-failed-tests-should-be-marked-as-Unstable-or-Failure-td3585663.html> > > > If that guidance doesn't meet your needs and you decide that you still > want to fail the build on test failures rather than mark the build > "Unstable" on test failures, then I think the easiest way is to invoke > mstest.exe directly (instead of using the MSTest plugin to launch it) and > then have the MSTest batch file exit with a non zero exit code if any test > fails. > > Mark Waite > > > > ------------------------------ > *From:* bearrito <j.barrett...@gmail.com > > *To:* jenkins...@googlegroups. com > *Sent:* Thursday, September 6, 2012 10:04 AM > *Subject:* MSTest Plugin doesn't fail the job on failing test. > > I would like the MSTest plugin to fail the job when a test fails. Is this > feasible ? > > > > > > > >
