> from the runner, something would still need to be able to aggregate those
> reports across multiple runs (ie: "ant test" being run in multiple dirs,
That's correct.
> Thinking about it purely from the point of view of the failure rate
> reports i've been generating, it would be nice if *every*
: first place. Some things were wrapped in "multiple-failure" exception,
: which didn't seem right to me. When I wrote the dedicated runner there
: was a lot of guessing involved, including XML reports (this bit not
: even part of JUnit, but Apache Ant and Apache Maven).
...
: It's
Hi Chris!
Right. All you say is true here. JUnit 4.x wasn't clear on the
specification of how such multiple failures should be handled in the
first place. Some things were wrapped in "multiple-failure" exception,
which didn't seem right to me. When I wrote the dedicated runner there
was a lot of
: bq. if the jenkins build re-runs a TestCase 5 times, there's still
: only one section for that suite
:
: I don't know about Jenkins XML, but this behavior is quite all right.
: Suite exceptions are either from static class init blocks or from
: "outside of test" scopes (like rule
Hi Chris.
bq. if the jenkins build re-runs a TestCase 5 times, there's still
only one section for that suite
I don't know about Jenkins XML, but this behavior is quite all right.
Suite exceptions are either from static class init blocks or from
"outside of test" scopes (like rule initialization,
FYI: I've made some improvements to my Jenkins Test reports recently...
http://fucit.org/solr-jenkins-reports/
Most significantly: I've added a new summary report called "Suspicious
Test Failure Rate"...
http://fucit.org/solr-jenkins-reports/suspicious-failure-report.html
This
