W make extensive use of the rollback pre test case paradigm for our proprietary application. It is invaluable. The test cases may run a little slower than if the setup/rollback was performed per suite, but the inter-testcase dependancies would be a nightmare for us to manage.
----- Original Message ----- From: "Scott Gray" <scott.g...@hotwaxmedia.com> To: dev@ofbiz.apache.org Sent: Thursday, December 24, 2009 3:14:26 AM GMT -05:00 US/Canada Eastern Subject: Re: Delegator rollback per test case? I'd really rather not have this idea die with a single response, I've done my best to describe the perceived benefits and it would be good to get some additional feedback from anyone who is interested. Thanks Scott On 13/12/2009, at 8:20 PM, Scott Gray wrote: > My intention was generally to have one test suite per sub-package > e.g. one for org.ofbiz.accounting.finaccount, one for > org.ofbiz.accounting.fixedasset, etc. > > Lets say I want to run 50 different order processing tests, > cancellations, edits, authorizations, packing, etc. To do this I > have to either create 50 different demo orders which is time > consuming or I have to split them up into different test suites > which will quickly leave us with a huge number of test suites all > doing very similar things. > > Aside from the above there are a few other things I don't like about > having only one rollback per test suite: > - When adding a new test case to a suite you have no idea what state > the database will be in unless you inspect all of the other test > cases in the suite and understand what they are doing. > - It makes test cases more difficult to maintain because you have no > idea where the dependencies are within a suite. With the change I'm > suggesting you would need to be explicit about dependancies by > putting the cases in a test group. > - If you make some application code or demo data changes and 10 test > cases in a suite start failing you have no idea if it's because a > single case is failing and the others depend on it or because your > changes are causing wide spread problems. Knowing the dependancies > helps the developer to understand how big the problem is before they > go through and examine every failed case. > > Ultimately my only goal here is to make writing tests easier, I'm > not approaching it with any idealistic "this is how tests should be > conducted" mentality. The easier we can make it, the more likely it > is that people will actually write them. > >> I'm unclear about something in your message though: are you saying >> you are doing it below and it's not meeting your needs, or that you >> would like to be able to do it as below? > > I'm not sure what you mean, what I'm suggesting below isn't > currently possible. The test-group tag actually does nothing at the > moment (other than add the child test cases to the suite, which is > the same as not having it at all). > > Regards > Scott > > On 13/12/2009, at 3:44 PM, David E Jones wrote: > >> >> My preference would still be to have one roll-back per test-suite, >> which basically means one transaction per XML file. If you want the >> tests to use the same data then put them in the same file. If you >> want them to use different data, then they go in their own files. >> >> I'm unclear about something in your message though: are you saying >> you are doing it below and it's not meeting your needs, or that you >> would like to be able to do it as below? >> >> -David >> >> >> On Dec 11, 2009, at 3:54 AM, Scott Gray wrote: >> >>> We've had this discussion before, here is last one I could find: >>> http://markmail.org/message/ftvs45vnzlobo7hb >>> >>> The solution I would like to propose for the problem David >>> described is for us to make more use of the test-group child >>> element of test-suite and have tests within those groups rollback >>> as a group, for all other test cases we'd roll them back as soon >>> as they complete. Here's an example of what I'm talking about >>> since that description probably did nothing for you: >>> >>> <test-suite> >>> >>> <test-case/> >>> <!-- Rollback here --> >>> <test-case/> >>> <!-- Rollback here --> >>> <test-case/> >>> <!-- Rollback here --> >>> >>> <!-- No rollbacks inside the test group --> >>> <test-group> >>> <test-case/> >>> <test-case/> >>> </test-group> >>> <!-- Rollback here --> >>> >>> </test-suite> >>> >>> A common problem I'm seeing is that we want to run multiple tests >>> on the same piece of demo data but can't because the previous test >>> has already altered it. An example might be where you want to try >>> shipping a single order in various different splits but you can't >>> do that at the moment without having a separate demo order for >>> each test. >>> >>> Thoughts? >>> >>> Thanks >>> Scott >>> >>> HotWax Media >>> http://www.hotwaxmedia.com >>> >>> >> >
