Hi Jeff. Danny is working with me. I am not sure it is going to be easier, because we will need two features currently not supported by the dynamic tests: 1. Combinatorics 2. Reading data from external data sources. Our goal is to utilize as much Gallio facilities as we can.
About controlling the ordering. We think to override the TestFixtureAttribute and TestAttribute. In our version of the TestFixtureAttribute we will populate our own context data structure which, among other things, will map each test method to the desired order ( or orders if the test is to be run several times). In the TestAttribute, we will override the Consume method and change the Order property before invoking base.Consume. If the test needs to be invoked several times with different orders, then our Consume override will run base.Consume several times with different values of the Order property (we have checked that invoking base.Consume N times duplicates the same test N times as well). This is the direction we are thinking. This way we let Gallio do all the hard work: 1. Combinatorics 2. Reading data sources 3. Running the tests in the right order. Do you see any principal issues with this plan? Thanks. P.S. This framework is part two in our overall testing effort. The first part is the framework I have built for testing different aspects of our entities, which projects dummy entity tests on real application entities. It works really well, but we now need to test more complex scenarios, where each scenario involves many individual tests. On 04/12/2009, at 01:03, Jeff Brown wrote: > It sounds like you might want to build up the test suite > dynamically instead of using ordinary test methods. It's a little > bit harder to do this way but you will have explicit control over > test ordering. > > Take a look at [DynamicTestFactory]. > > Jeff. > > On Wed, Dec 2, 2009 at 11:55 PM, Danny <[email protected]> wrote: > Dear sirs and ladies. > > We would like to have complex acceptance tests, which are composed of > individual test methods. The complex test defines the flow of the unit > test methods of which it is composed and this definition is read at > run-time from some data source (currently XML file). > > We envision the complex test as a test fixture which reads the flow > definition in its SetUp method, configures the flow of the test > methods and lets the test run. It is possible that the same test > method is run more than once. > > Of course, the tests are stateful, where any test method (but the > first) works on the state left by the previous one. (We are aware of > the Gallio.Ambient project and may be we wil consider it). > > Anyway, our immediate issue is that we do not know how to impose test > ordering based on some run-time specification. The Order attribute > seems to be inadequate for our needs, since it implies compile-time > test ordering specification. > > Any hints as to how to do it right are appreciated. > > Thanks. > > -- > > You received this message because you are subscribed to the Google > Groups "MbUnit.User" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to mbunituser > [email protected]. > For more options, visit this group at http://groups.google.com/ > group/mbunituser?hl=en. > > > > > -- > > You received this message because you are subscribed to the Google > Groups "MbUnit.User" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to mbunituser > [email protected]. > For more options, visit this group at http://groups.google.com/ > group/mbunituser?hl=en. ======================================================================== == There are two kinds of people. Those whose guns are loaded and those who dig. (The good, the bad and the ugly). So let us raise our cups for our guns always be loaded. -- You received this message because you are subscribed to the Google Groups "MbUnit.User" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/mbunituser?hl=en.
