Thanks Scott
On 13/12/2009, at 8:20 PM, Scott Gray wrote:
My intention was generally to have one test suite per sub-package e.g. one for org.ofbiz.accounting.finaccount, one for org.ofbiz.accounting.fixedasset, etc.Lets say I want to run 50 different order processing tests, cancellations, edits, authorizations, packing, etc. To do this I have to either create 50 different demo orders which is time consuming or I have to split them up into different test suites which will quickly leave us with a huge number of test suites all doing very similar things.Aside from the above there are a few other things I don't like about having only one rollback per test suite: - When adding a new test case to a suite you have no idea what state the database will be in unless you inspect all of the other test cases in the suite and understand what they are doing. - It makes test cases more difficult to maintain because you have no idea where the dependencies are within a suite. With the change I'm suggesting you would need to be explicit about dependancies by putting the cases in a test group. - If you make some application code or demo data changes and 10 test cases in a suite start failing you have no idea if it's because a single case is failing and the others depend on it or because your changes are causing wide spread problems. Knowing the dependancies helps the developer to understand how big the problem is before they go through and examine every failed case.Ultimately my only goal here is to make writing tests easier, I'm not approaching it with any idealistic "this is how tests should be conducted" mentality. The easier we can make it, the more likely it is that people will actually write them.I'm unclear about something in your message though: are you saying you are doing it below and it's not meeting your needs, or that you would like to be able to do it as below?I'm not sure what you mean, what I'm suggesting below isn't currently possible. The test-group tag actually does nothing at the moment (other than add the child test cases to the suite, which is the same as not having it at all).Regards Scott On 13/12/2009, at 3:44 PM, David E Jones wrote:My preference would still be to have one roll-back per test-suite, which basically means one transaction per XML file. If you want the tests to use the same data then put them in the same file. If you want them to use different data, then they go in their own files.I'm unclear about something in your message though: are you saying you are doing it below and it's not meeting your needs, or that you would like to be able to do it as below?-David On Dec 11, 2009, at 3:54 AM, Scott Gray wrote:We've had this discussion before, here is last one I could find: http://markmail.org/message/ftvs45vnzlobo7hbThe solution I would like to propose for the problem David described is for us to make more use of the test-group child element of test-suite and have tests within those groups rollback as a group, for all other test cases we'd roll them back as soon as they complete. Here's an example of what I'm talking about since that description probably did nothing for you:<test-suite> <test-case/> <!-- Rollback here --> <test-case/> <!-- Rollback here --> <test-case/> <!-- Rollback here --> <!-- No rollbacks inside the test group --> <test-group> <test-case/> <test-case/> </test-group> <!-- Rollback here --> </test-suite>A common problem I'm seeing is that we want to run multiple tests on the same piece of demo data but can't because the previous test has already altered it. An example might be where you want to try shipping a single order in various different splits but you can't do that at the moment without having a separate demo order for each test.Thoughts? Thanks Scott HotWax Media http://www.hotwaxmedia.com
smime.p7s
Description: S/MIME cryptographic signature
