On Mon, Jul 05, 2010 at 08:44:10PM +0100, Julio Merino wrote: > > The way some of the tests are organized suggests that the intended > > model is one test program per test victim (or per test victim and > > substantially different testing harness) and then one test case per > > bug affecting that victim. > > [...] > > For unit tests: you have a test program for every source module. Such > test program contains one unit test per function or one unit test per > function/subcases (e.g. error paths, an input array with elements, an > input array without elements, etc.) > > For integration/system/whatever tests: you have a test program per > "semantic" unit (I guess that's what you call victims ;-) and then > test cases for every particular scenario you want to represent.
That's not quite what I meant by "victim"; I meant "thing being tested", whether that's a library routine or a program or whatever. That is, you might have one test script for sort(1) and another compiled test program for, say, snprintf(3). If there's a lot of mechanism for making the test work, then you want to share it among all test instances that do the same thing. Conversely, however, it's silly to ram a bunch of unrelated code together into a single test program just because it all happens to be intended to test the same thing. So if you have multiple substantially different test harnesses, as might be the case for e.g. testing a filesystem in puffs and also in a native kernel running on qemu, that seems a good reason to have multiple test programs. However, my experience with test suites for a lot of projects (some of them highly nontrivial) is that it's always been sufficient for the test apparatus to run some collection of test programs, possibly on multiple inputs each, and compare the output to a corresponding known good version. This can easily be driven with make and doing so has always been perfectly satisfactory for me, so I don't claim to understand the fine points of the greater structure atf provides. -- David A. Holland dholl...@netbsd.org
