Chris Dolan wrote:
> On Dec 2, 2007, at 1:34 PM, nadim khemir wrote:
>> Because a TODO means that it is not done not: it might happend to be
>> done but
>> I'm not really sure, maybe I get lucky.
>>
>> Either one removes the TODO and all is fine. Or it might just be a
>> side effect
>> that you haven't planned that makes the test pass. Calling "unexpected
>> things" "features" does not make me feel more sure about quality.
(I missed the above part of the original post)
The red herring is "unexpected", and it's good that wording was removed from
TH 3. The feature is expected, but that it worked is unexpected. You're
talking like the TODO test is some slap-dash bit of code. The TODO test is
crafted to test a specific feature or bug, just like any other test.
What you're asking, essentially, is how do you know that the passing TODO test
is testing what it's supposed to test?
Well, remove "TODO" from that question and it reduces to the more general
question: how do you know a passing test is testing what it's supposed to
test? That is a question with plenty of answers.
>> This is exactely why I complained loud about Test::Exception::dies_ok.
>> It died
>> but not for the problem I expected. Other may be better or have more luck
>> than me though.
There's nothing wrong with dies_ok(), if I understand your complaint. It does
what it says on the tin, tests that the code died. And that's useful,
sometimes you don't know or don't care or can't accurately predict what the
exception will be. In these cases, without dies_ok() you'd have to fake up
some static exception for throws_ok().
This doesn't mean that people don't use dies_ok() when they should use
throws_ok(). Every tool is open to abuse. The solution is not to remove the
tool, but figure out why it's being abused. Maybe the answer is as simple as
putting throws_ok() first in the Test::Exception documentation?
As an analogy, look at ok() vs is() in Test::More. ok() just checks that the
statement is true, it doesn't care what it actually is. is() gets more
specific. In general, it's better to use is() but often you don't know (or
care) what the value actually is, just that it's true. Removing ok() from
Test::More would increase the resolution but also remove a lot of flexibility,
just as removing dies_ok() would.
>> I really would like to have -Werror for my tests.
A warning is "hey, you might have done something wrong!" A passing TODO test
is "hey, you might have done something right!" :)
--
If at first you don't succeed--you fail.
-- "Portal" demo
