On Feb 18, 2008 8:03 PM, Brett Porter <[EMAIL PROTECTED]> wrote: > On 19/02/2008, at 3:15 AM, John Casey wrote: > Things like: > - running the tests twice by necessity
Having being burned by this little foible... I often see people complain that the code coverage tools runs the unit tests twice. I often see people want to only run the tests once... We have had the following: * Unit tests that only pass on a multi-processor system when running with code coverage tools (due to synchronization problems. the code coverage's instrumented byte code by necessity has to synchronize updating method coverage, which forces the vm's to pick up some variable changes that it would not ordinarily see) * Unit tests that only fail when running with code coverage tools (one example is the tests we wrote after we found the synchronization logic error in our code) Then there is the fun of is the code coverage valid if the unit tests fail? I say that the coverage is valid if all the un-instrumented tests pass, but if the instrumented tests fail you need to examine the failures case by case. My rule of thumb is to _always_ run the tests twice, once with coverage and once without. The test failure during the second (instrumented) execution should not fail the build. - not being able to reuse the enhanced classes easily > - lack of consistency between build time checks and reporting >