Re: Why isn't Surefire more easier and more sensible to use?
I'm also looking for a way to get a surefire summary (like http://old.nabble.com/file/p26713465/1.txt this one ) to be created in a file, rather than in console only. I see it's not available today, am I right ? - -- Best regards, Evgeny http://evgeny-goldin.com/ -- View this message in context: http://old.nabble.com/Why-isn%27t-Surefire-more-easier-and-more-sensible-to-use--tp23611804p26713465.html Sent from the Maven - Users mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Why isn't Surefire more easier and more sensible to use?
Ive recently started using Surefire as part of a Maven project. I was surprised how inconvenient it was to use, and how things about it seemed, well... broken. 1. When running a suite of tests with surefire:test http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html, the main thing you want to see a summary of all failed tests. It is THE most important and normal thing to look at. I was surprised to discover Surefire doesnt produce this by default. Instead, it creates many individual reports, one per test, whether they passed or failed. 2. Then, one has to run a second plugin and special goal just to get the summary, which is what a typical user is going to be looking for. It puts the summary in a different directory from the individual files. And it doesnt produce this report as part of the normal build. 3. It generates the XML intermediate files that the report needs. It also generates identical txt files, for no apparent reason. And bizarrely, you can turn the XML off, but you cant turn the txt off. 4. Im posting to main Maven Users list as its 19 months since someone on the Surefire User list last received a reply. A pretty good imitation of /dev/null. Maybe time to close that list down? 5. You get some situations where because the tests fail, the build fails, and this actually stops the report generation which will tell you what test failed :( http://jira.codehaus.org/browse/SUREFIRE-507?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel http://www.eclipse.org/newsportal/article.php?id=216group=eclipse.technology.iam#216 The whole thing seems like a case of programmer-centric, not user-centric, thinking. This tool doesn't seem to have started with what a typical user would want and then worked backwards to meet this goal. Surefire is a widely used piece of infrastructure, and it should work better than it does. -Ben -- *Ben Hutchison Senior Developer * Level 2 476 St Kilda Road Melbourne VIC 3004 T 613 8807 5252 | F 613 8807 5203 | M 0423 879 534 | www.ibsglobalweb.com http://www.ibsglobalweb.com/ - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Why isn't Surefire more easier and more sensible to use?
Ive recently started using Surefire as part of a Maven project. I was really surprised how inconvenient it was to use, and how things about it seemed, well... stupid. 1. When running a suite of tests with surefire:test http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html, the main thing you want to see a summary of all failed tests. It is THE most important and normal thing to look at. I was surprised to discover Surefire doesnt produce this essential item by default. Instead, it creates many individual reports, one per test, whether they passed or failed. 2. Then, one has to run a second plugin and special goal just to get the summary, which is what any normal person is going to be looking for. It puts the summary in a different directory from the individual files. And it doesnt produce this report as part of the normal build. 3. It generates the XML intermediate files that the report needs. It also generates identical txt files, for no apparent reason. And bizarrely, you can turn the XML off, but you cant turn the txt off. And then you get weird situations where because the tests fail, the build fails, and this actually stops the report generation which will tell you what test failed :( http://jira.codehaus.org/browse/SUREFIRE-507?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel http://www.eclipse.org/newsportal/article.php?id=216group=eclipse.technology.iam#216 The whole thing seems like a classic case of programmer-centric, not user-centric, thinking. This tool doesn't seem to have started with what a typical user would want and then worked backwards to meet this goal. I dont particularly enjoy criticizing some programmer's unpaid contribution, but Surefire is a widely used piece of infrastructure, and it should work better than it does. -Ben -- *Ben Hutchison Senior Developer * Level 2 476 St Kilda Road Melbourne VIC 3004 T 613 8807 5252 | F 613 8807 5203 | M 0423 879 534 | www.ibsglobalweb.com http://www.ibsglobalweb.com/
Re: Why isn't Surefire more easier and more sensible to use?
On 19/05/2009, at 11:50 AM, Ben Hutchison wrote: Ive recently started using Surefire as part of a Maven project. I was surprised how inconvenient it was to use, and how things about it seemed, well... broken. 1. When running a suite of tests with surefire:test http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html , the main thing you want to see a summary of all failed tests. It is THE most important and normal thing to look at. I was surprised to discover Surefire doesnt produce this by default. Instead, it creates many individual reports, one per test, whether they passed or failed. This is the output from one of my projects: [INFO] [surefire:test] [INFO] Surefire report directory: /Users/brett/scm/archiva/archiva/ archiva-modules/archiva-base/archiva-common/target/surefire-reports --- T E S T S --- Running org.apache.maven.archiva.common.utils.BaseFileTest Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.05 sec Running org.apache.maven.archiva.common.utils.VersionComparatorTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 sec Running org.apache.maven.archiva.common.utils.DateUtilTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 sec Running org.apache.maven.archiva.common.utils.PathUtilTest Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec Results : Tests run: 15, Failures: 0, Errors: 0, Skipped: 0 What are you looking for that is different to that? 2. Then, one has to run a second plugin and special goal just to get the summary, which is what a typical user is going to be looking for. It puts the summary in a different directory from the individual files. And it doesnt produce this report as part of the normal build. I assume you are talking about the HTML report. That is in a separate plugin because it is a separate function and people just wanting the above don't want to drag down the report generation dependencies. 3. It generates the XML intermediate files that the report needs. It also generates identical txt files, for no apparent reason. Because the XML isn't very readable by humans. I always look at the text files for failures. And bizarrely, you can turn the XML off, but you cant turn the txt off. http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html#useFile (yes, naming could be more consistent, the Xml one was probably added later) 4. Im posting to main Maven Users list as its 19 months since someone on the Surefire User list last received a reply. A pretty good imitation of /dev/null. Maybe time to close that list down? yes, we already indicated they'd fold into the main maven lists, just haven't got around to doing it. 5. You get some situations where because the tests fail, the build fails, and this actually stops the report generation which will tell you what test failed :( http://jira.codehaus.org/browse/SUREFIRE-507?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel http://www.eclipse.org/newsportal/article.php?id=216group=eclipse.technology.iam#216 That might be a regression in the embedded version of Maven. Cheers, Brett - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Why isn't Surefire more easier and more sensible to use?
brettporter wrote: On 19/05/2009, at 11:50 AM, Ben Hutchison wrote: 1. When running a suite of tests with surefire:test http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html , the main thing you want to see a summary of all failed tests. It is THE most important and normal thing to look at. I was surprised to discover Surefire doesnt produce this by default. Instead, it creates many individual reports, one per test, whether they passed or failed. This is the output from one of my projects: [INFO] [surefire:test] ... snip ... Tests run: 15, Failures: 0, Errors: 0, Skipped: 0 What are you looking for that is different to that? Hi Brett, Thanks for responding. To answer your question, I would like a file created that contains a test summary like the above, by default. Because (a) the Maven console can become very crowded and noisy, and (b) in embedded situations, the console may not be obviously visible. In my usage yesterday (where I had a frustrating experience) both (a) and (b) applied. -Ben -- View this message in context: http://www.nabble.com/Why-isn%27t-Surefire-more-easier-and-more-sensible-to-use--tp23611804p23628539.html Sent from the Maven - Users mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
