[ http://jira.codehaus.org/browse/SUREFIRE-457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Fabulich updated SUREFIRE-457: ---------------------------------- Attachment: testNGListenerTest.zip Jason Chaffee claims that you can get the right thing with onStart/onFinish notifications. I've attached a sample project that seems to show that this is false. http://markmail.org/message/in2iosogjpjev5yk http://markmail.org/message/yhbzjbhxmcydvqna I've attached a sample project that simply attaches an ITestListener that logs to system.out whenever it gets called, specifically during onStart and onFinish. It's got two test classes: Foo and Bar, each of which have one method "foo()" and "bar()" respectively. So you'd expect: onStart [Foo] onTestStart [foo()] onTestFinish [foo()] onFinish [Foo] onStart [Bar] onTestStart [bar()] onTestFinish [bar()] onFinish [Bar] But here's what you get instead: onStart onTestStart [foo()] onTestFinish [foo()] onTestStart [bar()] onTestFinish [bar()] onFinish ... not too useful, is it? In fact, onStart is called at the start/end of the "test" which is TestNG's very silly term for a collection of tests that isn't a "group" or a "suite". > TestNG should log on the console before/after every test class > -------------------------------------------------------------- > > Key: SUREFIRE-457 > URL: http://jira.codehaus.org/browse/SUREFIRE-457 > Project: Maven Surefire > Issue Type: Wish > Components: TestNG support > Affects Versions: 2.4, 2.4.1, 2.4.2 > Reporter: Dan Fabulich > Priority: Minor > Fix For: Future > > Attachments: testNGListenerTest.zip > > > In Surefire 2.3.x, when running the tests in "directory mode" (i.e. without a > suite.xml file) we used to run each TestNG class as a separate suite. As a > result, we logged to the console before/after every test class. > This behavior totally broke tests that had rich interdependencies (which is a > big part of the point of TestNG), so in Surefire 2.4 we handed more control > over to TestNG. As a result, we let TestNG run the entire directory at once, > and we aren't notified when classes start/end. Since we aren't notified, we > can't log to the console before/after every test class. But this looks like > a regression to those who were used to the 2.3.x behavior. > (This is trickier than it looks, because TestNG can/will run test methods > entirely out of order, running part of class X, and then part of class Y, > then back to class X, then a bit of class Z, etc. And that's not even > considering parallelized testing.) > I argue that to fix this "right" we'd need TestNG to notify us when classes > start/end; we should pressure the TestNG team to provide this functionality. > Benjamin has argued that there should be an option to run each test in its > own suite; I think that option is confusing and error-prone. > http://www.nabble.com/Re%3A-Test-Suites%2C-Ant%2C-Surefire%2C-and-JunitReport-p15094586.html > If you think you want a one-class-per-suite option, I offer this remark: Many > TestNG tests aren't safe to run in one-class-per-suite mode. If your tests > are safe to run in that mode, then they're also safe to run in JUnit 4. (In > fact, if your tests are that simple, you probably aren't using any of > TestNG's unique features, and you should prefer to use JUnit 4.) So, as a > workaround, don't use TestNG; use JUnit 4 instead. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira