On Fri, Dec 18, 2009 at 06:50:59PM +1030, David Kettler wrote: > > Nice idea, but let me play devil's advocate for a second, to make sure we > > are clear-headed about this. Just to go over our terms, our smallest > > primitive is the "test", and tests are grouped in "suites". What we are > > really getting at here is adding a new type---a group of > suites---which on > > the surface sounds self-contradictory. Is there a parallel for this in > > other testing frameworks, or did we fail in some basic assumption about > > how testing should work, that we find ourselves needing to reinvent unit > > testing? > > I don't have much experience with other widely used test frameworks, > but this does seem reasonable. The primitive "test" tests some > individual aspect of the program; the test "suite" aggregates the test > results for a unit of the program. We then want to be able to > aggregate the results of the unit test suites for the whole program. > Note that the way I've extended walnut_totals is (at least partially) > recursive, so we won't need to keep adding layers. > > Here's a couple of examples that have something similar to this extra > layer. > > The git test suite is structured rather like the conkeror tests in > that groups of tests are written in files that can either be run > individually, or all the tests can be run, with an aggregation of the > results. > > http://git.kernel.org/gitweb.cgi?p=git/git.git;a=blob;f=t/README;hb=master > http://git.kernel.org/gitweb.cgi?p=git/git.git;a=blob;f=t/aggregate-results.sh;hb=master > > > In cunit one writes tests in a suite and register the suites with a > test registry. > > http://cunit.sourceforge.net/doc/introduction.html#usage > > regards, David.
Hi David, As I think about this, I don't think that program-termination is really the right time to output a testing summary. It only appears to be the right time because the typical way in which we run tests is with the -l switch from the command line in a batch invocation of a fresh instance of conkeror, but it is clearly not the right time if conkeror is not immediately exited after testing. Instead, I would propose a command-line switch called -walnut-summary (or -walnut-summarize?), to give the user true control over the time of summary generation, without introducing new assumptions about the manner in which tests will be loaded. What say you? -- John Foerch _______________________________________________ Conkeror mailing list [email protected] https://www.mozdev.org/mailman/listinfo/conkeror
