On 21.02.2011 13:12, Philip Martin wrote: > Branko Čibej <br...@e-reka.si> writes: > >> We should not be optimising tests for performance over clarity, ever. In >> other words -- don't combine them. Anyone who has trouble with tests >> taking too long can use a RAMdisk, --bdb-txn-nosync, and other tweaks to >> make the tests run faster. > I do that. The time to run the testsuite keeps growing. > >> I think you've all gone way off track here. Reality check, please? What >> is the purpose of tests? To validate the behaviour of the code, or to >> make developers' life easier? > To make it easy for developers to validate the code :) > > Like all code it requires a judgement about how it should be organised. > I agree that complicated tests should be separate, but I think checking > that: > > "svn revert foo" > > and > > "svn resoved foo" > > both report "foo is not a local path" doesn't require two completely > separate tests. > > How far do you think we should go? A dozen separate tests for revert > alone: > > "svn revert versioned_and_modified" > "svn revert versioned_and_unmodified" > "svn revert url" > "svn revert unversioned" > "svn revert versioned_and_modified versioned_and_unmodified" > "svn revert versioned_and_unmodified unversioned" > etc
It's certainly easier to read the test output that way. :) You could argue that tiny tests that do not modify a sandbox can be merged together. Of course, if there's a bug in the implementation and one of those subtests /does/ modify the sandbox, causing another down the line to fail for no obvious reason, that hardly makes life easier for the poor sod who has to analyse that mess. There's a balancing act here. Talk about a "global sandbox" reused by several test cases definitely makes my hair stand on end and I do recommend against that. Combining several small tests into a single testcase is a different and slightly more palatable approach, as long as everyone agrees to be the poor sod that has to analyse the fallout. :) -- Brane
