On Saturday 06 November 2010 13:05:13 Jérôme M. Berger wrote: > spir wrote: > > On Sat, 6 Nov 2010 01:21:25 -0700 > > > > Jonathan M Davis <jmdavisp...@gmx.com> wrote: > >> I believe strongly that a unit test block which has a failure should end > >> excecution. For many such tests, continuing would be utterly pointless, > >> since each successive test relies on the last. > > > > I don't understand. I can have one dozen test cases for each of one dozen > > funcs. All 144 tests are independant. I prefere the possibility to see > > all test errors in one go, if any. Anyway, there may be a flag > > STOP_AT_FIRST_TEST_ERROR (or the opposite). > > In that case, you should have a dozen dozen different test blocks, > each for a single test case of a single function. Under Jonathan's > proposal, when a test block fails, it would stop the current *block* > (i.e. test case) but allow the remaining blocks to be run.
Exactly. In every other unit testing framework that I've used, a unit test is a function, and when that function fails, that test fails and does not continue to be executed. Subsequent unit test functions are run, but once a particular function fails, it does not continue. If you have a whole lot of unrelated asserts in a single function, I could see how it could be annoying to have the function stop on one failure, but there are plenty of cases where you have more complicated tests which aren't just a bunch of asserts on single function calls to check the result, and in those sorts of tests, the state at each assert relies on the state at the last assert being correct, so once one fails, all subsequent asserts mean nothing. So, what you do if you have a lot of indepent asserts, and you want more of them to run if one fails is that you break them up into multiple unit test functions - or in the case of D, more unit test blocks. There was a rather long discussion on this on the Phobos list a few months back ( you should probably start reading it here if you really want to read it: http://is.gd/gN1fq ) after Walter had made it so that all unit tests continued even after a failed assert. It was decided that that was a _bad_ idea, and ultimately, I believe that the decision was that each unittest block in a module would be run independently, but that upon any failure in a unittest block, that block would terminate. However, there are changes that must be made in the compiler to make it possible to make it so that subsequent unittest blocks in a module run after a failure in one, so at the moment, if you get a failure in a unittest block, none of the other unittest blocks in that module run (though the unittest blocks in other modules still run). The change should be made eventually, however. Actually, I should probably make sure that it's in bugzilla so that it isn't forgotten. - Jonathan M Davis