On Fri, Apr 02, 2004 at 02:51:11PM -0600, Andy Lester wrote: > > coded correctly. So it's desirable to see the results of the lower level > > tests first because running the higer level tests could be a waste of time. > > But how often does that happen? Why bother coding to optimize the > failures? > > Besides, if you have a smokebot to run the tests for you, then you don't > care how long things take.
It's more the time spent looking at the test results rather than the time spent running the tests. So actually it's the result presentation order that matters. Basically you want to consider the failure reports starting from the lowest level as these may make the higher level failures irrelevant. The order the tests actually ran in should be irrelevant to the outcome but if you're running from the command line the run order determines the presentation order. Even if you have a smoke bot, you presumably run the tests (depends on the size of the suite I suppose) before a checkin and it's convenient to know that the first failure message you see if the most relevant (ie at the lowest level). Also when running tests interactively it's nice to be able to save even 30 seconds by killing the suite if a low level test fails, F
