Ive recently started using Surefire as part of a Maven project.
I was really surprised how inconvenient it was to use, and how things
about it seemed, well... stupid.
1. When running a suite of tests with "surefire:test
<http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html>",
the main thing you want to see a summary of all failed tests. It is THE
most important and normal thing to look at. I was surprised to discover
Surefire doesnt produce this essential item by default. Instead, it
creates many individual reports, one per test, whether they passed or
failed.
2. Then, one has to run a second plugin and special goal just to get the
summary, which is what any normal person is going to be looking for. It
puts the summary in a different directory from the individual files. And
it doesnt produce this report as part of the normal build.
3. It generates the XML intermediate files that the report needs. It
also generates identical txt files, for no apparent reason. And
bizarrely, you can turn the XML off, but you cant turn the txt off.
And then you get weird situations where because the tests fail, the
build fails, and this actually stops the report generation which will
tell you what test failed :(
http://jira.codehaus.org/browse/SUREFIRE-507?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel
http://www.eclipse.org/newsportal/article.php?id=216&group=eclipse.technology.iam#216
The whole thing seems like a classic case of programmer-centric, not
user-centric, thinking. This tool doesn't seem to have started with what
a typical user would want and then worked backwards to meet this goal.
I dont particularly enjoy criticizing some programmer's unpaid
contribution, but Surefire is a widely used piece of infrastructure, and
it should work better than it does.
-Ben
--
*Ben Hutchison
Senior Developer
* Level 2 476 St Kilda Road Melbourne VIC 3004
T 613 8807 5252 | F 613 8807 5203 | M 0423 879 534 |
www.ibsglobalweb.com <http://www.ibsglobalweb.com/>
