On Mon, Oct 1, 2012 at 3:15 PM, Baptiste MATHUS <bmat...@batmat.net> wrote:
> I won't comment much on the smell of having tests that you wanna ignore > even in CI... > It's not extremely pretty, but to clarify, these tests are all passing and not-ignored in the mainline code. It's just with this new, unfinished switch that we need some of them temporarily disabled. This is a separate job -- the one that gates production changes requires all these tests to pass (which is indeed why we're not just marking them with @Ignore). Even though, what you explain seems quite complicated just to ignore tests > failures. > Did you consider just using Maven Surefire testFailureIgnore parameter? > http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html#testFailureIgnore > That doesn't seem to work. The mvn stdout reports success, but the job is still marked as unstable. Yuval > 2012/10/1 Yuval Shavit <ysha...@akiban.com> > >> Hi all. I have a job with a maven run in which I expect test failures, >> and I want the job to *not* be marked as UNSTABLE if there are failures; >> I'll be doing some custom analysis in script tasks after the maven task in >> which I'll determine the build's status. In other words, what I'd like is: >> >> 1) mvn test ... >> 2) reset build.result to SUCCESS >> 3) custom script, which may mark status as FAILING >> >> I tried putting a groovy system script at step (2) which sets >> build.result to Result.SUCCESS, but I wasn't able to get that to work. The >> script runs correctly (I had it put a badge on the job as a test), but the >> status stays as UNSTABLE is there are any unit test failures. I also tried >> using reflection to set the Run.result field directly, to no avail. >> >> My current approach is to have two jobs. The "kickoff" job has a groovy >> script which invokes the "main" job, which does the actual testing. Once >> that main job is done, the groovy script marks the kickoff job as failing >> if the main one failed, and as passing if the main job succeeded or was >> unstable. This works, but it's clumsy and not easy to drill down to the >> actual failures from the kickoff job. >> >> Btw, I realize that JUnit tests can expect errors, but that's not what I >> want. The project I'm working on is testing an experimental switch which is >> not done and thus causes a lot of tests to fail; I basically want to ensure >> that there are no regressions among those tests which we know have passed >> in the past, while allowing tests to fail if they always have. I'm open to >> suggestions besides my approach above, but I'd like to avoid completely >> rewiring our tests for this. >> >> Thanks! >> Yuval >> > > > > -- > Baptiste <Batmat> MATHUS - http://batmat.net > Sauvez un arbre, > Mangez un castor ! >
