+1 - this needs to happen - the rollback is invaluable as Marc said. Cheers, Ruppert -- Tim Ruppert HotWax Media http://www.hotwaxmedia.com
o:801.649.6594 f:801.649.6595 On Dec 24, 2009, at 5:26 AM, Marc Morin wrote: > W make extensive use of the rollback pre test case paradigm for our > proprietary application. It is invaluable. The test cases may run a little > slower than if the setup/rollback was performed per suite, but the > inter-testcase dependancies would be a nightmare for us to manage. > > > ----- Original Message ----- > From: "Scott Gray" <scott.g...@hotwaxmedia.com> > To: dev@ofbiz.apache.org > Sent: Thursday, December 24, 2009 3:14:26 AM GMT -05:00 US/Canada Eastern > Subject: Re: Delegator rollback per test case? > > I'd really rather not have this idea die with a single response, I've > done my best to describe the perceived benefits and it would be good > to get some additional feedback from anyone who is interested. > > Thanks > Scott > > On 13/12/2009, at 8:20 PM, Scott Gray wrote: > >> My intention was generally to have one test suite per sub-package >> e.g. one for org.ofbiz.accounting.finaccount, one for >> org.ofbiz.accounting.fixedasset, etc. >> >> Lets say I want to run 50 different order processing tests, >> cancellations, edits, authorizations, packing, etc. To do this I >> have to either create 50 different demo orders which is time >> consuming or I have to split them up into different test suites >> which will quickly leave us with a huge number of test suites all >> doing very similar things. >> >> Aside from the above there are a few other things I don't like about >> having only one rollback per test suite: >> - When adding a new test case to a suite you have no idea what state >> the database will be in unless you inspect all of the other test >> cases in the suite and understand what they are doing. >> - It makes test cases more difficult to maintain because you have no >> idea where the dependencies are within a suite. With the change I'm >> suggesting you would need to be explicit about dependancies by >> putting the cases in a test group. >> - If you make some application code or demo data changes and 10 test >> cases in a suite start failing you have no idea if it's because a >> single case is failing and the others depend on it or because your >> changes are causing wide spread problems. Knowing the dependancies >> helps the developer to understand how big the problem is before they >> go through and examine every failed case. >> >> Ultimately my only goal here is to make writing tests easier, I'm >> not approaching it with any idealistic "this is how tests should be >> conducted" mentality. The easier we can make it, the more likely it >> is that people will actually write them. >> >>> I'm unclear about something in your message though: are you saying >>> you are doing it below and it's not meeting your needs, or that you >>> would like to be able to do it as below? >> >> I'm not sure what you mean, what I'm suggesting below isn't >> currently possible. The test-group tag actually does nothing at the >> moment (other than add the child test cases to the suite, which is >> the same as not having it at all). >> >> Regards >> Scott >> >> On 13/12/2009, at 3:44 PM, David E Jones wrote: >> >>> >>> My preference would still be to have one roll-back per test-suite, >>> which basically means one transaction per XML file. If you want the >>> tests to use the same data then put them in the same file. If you >>> want them to use different data, then they go in their own files. >>> >>> I'm unclear about something in your message though: are you saying >>> you are doing it below and it's not meeting your needs, or that you >>> would like to be able to do it as below? >>> >>> -David >>> >>> >>> On Dec 11, 2009, at 3:54 AM, Scott Gray wrote: >>> >>>> We've had this discussion before, here is last one I could find: >>>> http://markmail.org/message/ftvs45vnzlobo7hb >>>> >>>> The solution I would like to propose for the problem David >>>> described is for us to make more use of the test-group child >>>> element of test-suite and have tests within those groups rollback >>>> as a group, for all other test cases we'd roll them back as soon >>>> as they complete. Here's an example of what I'm talking about >>>> since that description probably did nothing for you: >>>> >>>> <test-suite> >>>> >>>> <test-case/> >>>> <!-- Rollback here --> >>>> <test-case/> >>>> <!-- Rollback here --> >>>> <test-case/> >>>> <!-- Rollback here --> >>>> >>>> <!-- No rollbacks inside the test group --> >>>> <test-group> >>>> <test-case/> >>>> <test-case/> >>>> </test-group> >>>> <!-- Rollback here --> >>>> >>>> </test-suite> >>>> >>>> A common problem I'm seeing is that we want to run multiple tests >>>> on the same piece of demo data but can't because the previous test >>>> has already altered it. An example might be where you want to try >>>> shipping a single order in various different splits but you can't >>>> do that at the moment without having a separate demo order for >>>> each test. >>>> >>>> Thoughts? >>>> >>>> Thanks >>>> Scott >>>> >>>> HotWax Media >>>> http://www.hotwaxmedia.com >>>> >>>> >>> >> >
smime.p7s
Description: S/MIME cryptographic signature
