Smylers wrote:
Bram writes:
At the moment foo() returns 3.
Time passes and code changes.
Now there are 3 options:
foo() returns 1, this will result in 'unexpected todo test passed'
being outputted;
foo() returns 3, no special output is produced;
foo() returns 4, no special output is produced;
To me the last one seems like a limitation.
The result of foo() is now different then when the TODO test was
written but there is nothing in the output that indicates this.
As I understand it, you want to know when broken code breaks differently.
I can sort of see the point as a regression test... sort of. But if you're at
the point of fussiness that broken code has to break in a specific way... is
it broken any more? I agree with others on this list, if you expect it to
return 3, put in a test that says it should return 3.
As a technique, paying attention to how broken code changes, why does it
matter that broken code breaks differently? What does this information tell
you that might fix code? Is the opportunity cost of paying attention to and
maintaining specific tests for broken code worthwhile?
It seems like a dodgey regression to me of the "something changed but we don't
know if its significant" variety that generates so many false alarms and so
much busy work keeping it up to date. A classic example is a test that does a
simple string compare with a big woodge of HTML. Any inconsequential change
to the formatting is going to trip the test leading to many false negatives
and eventually ignoring the test.
--
91. I am not authorized to initiate Jihad.
-- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
http://skippyslist.com/list/
