Subject: Re: Sparse Test Output
From: Paul Beckingham <[EMAIL PROTECTED]>
Date: Thu, 26 Oct 2006 10:58:46 -0400

}There is no loss of information, but there is a saving as N approaches and
}exceeds 4.  Less so with test names, diag output etc.

Assuming all tests are run, and there are no extra tests, you'd be right.

Consider:

 1..4
 ok 1
 not ok 2
 ok 3

versus:

 1..4 sparse
 not ok 2

The "sparse" would make a parser assume that every test not shown passed.  
However, as you can see, test 4 was not run.  That is a failure in my 
book, but with "sparse", you'd never catch it.

Alternately, what would happen if "ok 4" and "ok 5" showed up?  That's an 
extra test - again, a failure that "sparse" wouldn't catch.

Since the harness shouldn't alter the test stream, you wouldn't want to 
have the harness be the one to inject a "not ok 4 - test not run" to catch 
the first case and/or display tests outside the plan.

Also, how would sparse affect TODO and SKIP?  Seems to me you'd want a 
TODO or SKIP to always show.

Tests, in my book, aren't and shouldn't be about efficiency, unless 
efficiency is what you're testing.  They should be about correctness, then 
completeness, then efficiency after that.

-Pete K
-- 
Pete Krawczyk
  perl at bsod dot net

Reply via email to