Smylers wrote:
Bram writes:

At the moment foo() returns 3.

Time passes and code changes.

Now there are 3 options:

foo() returns 1, this will result in 'unexpected todo test passed' being outputted;
foo() returns 3, no special output is produced;
foo() returns 4, no special output is produced;

To me the last one seems like a limitation.
The result of foo() is now different then when the TODO test was written but there is nothing in the output that indicates this.

As I understand it, you want to know when broken code breaks differently.

I can sort of see the point as a regression test... sort of. But if you're at the point of fussiness that broken code has to break in a specific way... is it broken any more? I agree with others on this list, if you expect it to return 3, put in a test that says it should return 3.

As a technique, paying attention to how broken code changes, why does it matter that broken code breaks differently? What does this information tell you that might fix code? Is the opportunity cost of paying attention to and maintaining specific tests for broken code worthwhile?

It seems like a dodgey regression to me of the "something changed but we don't know if its significant" variety that generates so many false alarms and so much busy work keeping it up to date. A classic example is a test that does a simple string compare with a big woodge of HTML. Any inconsequential change to the formatting is going to trip the test leading to many false negatives and eventually ignoring the test.


--
91. I am not authorized to initiate Jihad.
    -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
           http://skippyslist.com/list/

Reply via email to