On 25 Jun 2004, at 16:10, Andrew Pimlott wrote: [snip]
I thought the "isolation" principle that people were talking about is
that before every test, a "setup" method is called, and after every test
a "teardown" is called, automatically by the test harness. This
seems to require one method == one test.
It doesn't. I think there's a confusion of vocabulary.
In the Perl world 'test' refers to something like 'is' and 'ok' from Test::More and friends. We're interested in the number of successful tests.
In the xUnit world these are called assertions, not a tests, and in general we're /not/ concerned with the number of assertions that succeed.
In the xUnit world a /test/ is a method with one or more assertions that checks one particular bit of behaviour in the code. A failed test is a method where one assertion failed. A passed test is a method where all assertions succeed.
From this perspective it makes sense to abort after the first assertion has failed - since the 'test' has failed. Think of it like logical '&&' short circuiting, or maybe like SKIP blocks (there's no point doing the other assertions because the thing we're testing has already failed).
Test isolation is the idea each test (not assertion) can run independently from every other. This means when we have a failure we can quickly focus in on exactly what caused the problem. We can be confidant that four test failures indicates four separate problems, not a cascade of failure within the test suite itself.
There is a school of thought that one-assertion-per-test is a good goal to aim for, but not everybody agrees. For some discussion see:
- <http://www.testdriven.com/modules/newbb/viewtopic.php? viewmode=flat&topic_id=363&forum=6>
- <http://www.artima.com/weblogs/viewpost.jsp?thread=35578>
Hopefully this makes some vague sort of sense.
Cheers,
Adrian
