On Fri, Sep 27, 2013 at 11:42:10AM +0200, Jens Rehsack wrote: > Am 27.09.2013 um 00:03 schrieb Tim Bunce <tim.bu...@pobox.com>: > > > On Wed, Sep 25, 2013 at 08:36:22PM +0200, Jens Rehsack wrote: > > > >> For me, a G10 is interesting (handling SQL::Statement as a DBD) > >> > >> G10: Provide tools/infrastructure to combines several test cases in > >> particular order into tests to allow workflows being tested > >> and stress tests by looping can be performed. > > > > Also here. I'm not quite sure what problem this is trying to address. > > > This is more or less an idea I had during test development. For a lot > of tests, we write some sequences of statements to finally prove one > or a few things working fine (at least for DBD::CSV, DBD::DBM and a > lot of SQL::Statement tests - but IIRC, this is true for SQLite, too). > > So for testing whether UPDATE/DELETE works, we create some tables, do > some inserts and after that, modify them. For SELECT testing, same. > For ChopBlank testing (via select, of course), similar. For JOIN testing, > more tables - but finally the same. This goes on … (different edge cases > to test, but finally all the same). > > For SQL::Statement I added a performance check )for being run through > the profilers we have) which is assembled from one or more of the above > tests. But by hand. > > If there is a design of combining test cases to a sequence of tests (eg. > create tables -> fill data -> select -> update -> delete -> drop), it > could help simplifying test cases and allow put loops around sequences > to redo some test sequence for 100 times for profiling. > > The design finally choosen for this goal might be a composite for test > cases - but the test case design shall allow doing that.
Test files need to be self-contained, so they can be run in parallel and in randomized order. They'd naturally have a setup-fixture phase, a run-tests phase, and a teardown-fixture phase. Repeating the whole sequence of phases N-times would be easy enough. For some, but not all tests, repeating the run-tests phase N-times between the setup and teardown phases would be possible, and faster. I think this ability would fall-out nicely from defining a suitable API to the code in the test modules. Then it's a simple change to the code that calls the subs/methods in the test modules. Tim. p.s. I'll add parallel running to the requirements - it has significant implications on the design, e.g., unique names for tables.
