It comes down to this:

SKIP tests are for when it'll never work.  Or if you don't care.

TODO tests allow the author to be notified when something that was broken 
starts working again.  It doesn't matter what that thing is.

If a module you depend on breaks and you want to know when it works again so 
you can start depending on it again, use a TODO test.  While its broken treat 
it like any other known bug (wrap it in a TODO test) or fix it by using 
something else.  Its not the end of the world if some users get an 
"unexpectedly succeeded" when it starts working again (though the Test::Harness 
output for bonus TODO tests could use some work to make it look less like a 
failure).

When the TODO test starts passing you can up your MakeMaker version dependency 
on that module and remove the TODO.


Let me give a real world example.

I recently worked in a shop where lots of modules were out of date.  For 
example, they were using CGI.pm 2.x.  I wanted to upgrade to the latest for 
some features, bug fixes, etc.  I asked if there was any reason why not and got 
some "oh, don't do that.  It'll break things."  "Like what?", I asked.  "I 
don't remember.  We tried it a few years ago."

If, when someone encountered a CGI.pm bug they threw in a TODO test for it I'd 
have known what the problems were.  As it was all I could do is upgrade, run 
the tests and hope they were thorough enough to catch whatever problem CGI.pm 
caused.

Another example, same shop.  They're using 5.6.  I'm trying to upgrade to 5.8 
but all sorts of things are failing.  One issue is with their Unicode code.  
They have all sorts of Unicode code to work around 5.6 weirdness.  Now that I"m 
using 5.8 I'm getting test failures.  Is it because they're relying on 5.6 bugs 
or because things are really broken?  Is a given bit of weird code in there to 
work around a 5.6 bug or does it really need to work that way?  I don't know.

If people had written TODO tests for the perl bugs they encountered in 5.6 I'd 
have known what was a bug workaround and what was a real problem.

I never did manage to get 5.8 working.

Reply via email to