"Gwern Branwen" <[email protected]> writes: > On Tue, Dec 23, 2008 at 8:00 AM, Florent Becker wrote: >> Hi all, >> >> instead of using alphabetical order, would it make sense to order the >> unit tests by "difficulty", or "fundamentalness". This way, when many >> tests fail, the first one to fail would be the one to look at >> first. This would be done by appending a number to the name of the test, >> from 00-trivial-test.sh to 99-corner-case.sh. What do you think? >> >> Florent > > I think that one probably could not order all tests by > difficulty/fundamentalness
It seems reasonable to me to at least categorize tests into a handful of classes, e.g. "internals", "user interface", "network". We already have a separate "network" class! > (switching to random order broke some tests, IIRC). Merely *reversing* the order in which tests were done exposed two setup/teardown bugs. What *should* happen, but I haven't coded for yet, is for the test harnes to essentially set up a scratch area per test, run the test in it, and teardown the scratch area if the test passes. This way it should be difficult for tests to interfere with one another -- they would need to intefere outside their working directories (e.g. $HOME/.darcs/cache, if $HOME not also set to the scratch area). _______________________________________________ darcs-users mailing list [email protected] http://lists.osuosl.org/mailman/listinfo/darcs-users
