Quoting Smylers <[EMAIL PROTECTED]>:
Bram writes:
At the moment foo() returns 3.
Time passes and code changes.
Now there are 3 options:
foo() returns 1, this will result in 'unexpected todo test passed'
being outputted;
foo() returns 3, no special output is produced;
foo() returns 4, no special output is produced;
To me the last one seems like a limitation.
The result of foo() is now different then when the TODO test was
written but there is nothing in the output that indicates this.
If you believe that (until the TODO is done) foo will consistently
return 3,
I believe that at the moment it returns 3 and that when another, at
first sight, unrelated bug gets solved it starts returning something
else than 3.
and you wish to be alerted if it suddenly starts returning 4,
then surely you can do that with a non-TODO test which checks for its
being 3?
Then what happens if it starts returning 4?
Then the test script will report a FAIL, and users will/might start
ignoring failures.
Which is a bad thing (IMHO).
The todo test indicates that something doesn't behave as it should.
If it suddenly starts returning another value, which is still just as
wrong, then it shouldn't result in FAIL.
Incorrect? Yes
Not expected? Yes
Fail? No since that particular code is known to misbehave (in what way
it misbehaves can (and will) change over time).
In the particular test foo() could get altered in quite a lot of ways
without realizing that it also affects the behavior of foo().
This could be a good change (< 3) or a bad one (> 3) but bottom line
it means that things changed that weren't expected and that, most
likely, means that there are tests missing to cover all the behavior
of the change.
Kind regards,
Bram
