Bryce McKinlay wrote:
It is not reasonable to expect test case developers ensure that all
tests "run linearly". Exceptions can potentially be thrown at any
time, so to ensure linearity, every check() call would need wrapped
with a try/catch.
If there is a linear sequence of checks in a test() method, and an
(unexpected) exception causes the test() method to exit early (after,
say, 3 checks out of 10), I don't consider that to be non-linear. If we
are comparing run A to run B, and 10 checks complete in run A, but only
3 checks complete in run B, we can safely assume that checks 4 to 10
were not completed in run B, and report that.
The majority of test() methods in Mauve are written that way, so I don't
think it is an unreasonable requirement, especially if it means we can
develop better comparison/regression reporting on top of the existing
TestHarness.
Regards,
Dave
