Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?
Chris Dolan wrote: > On Dec 2, 2007, at 1:34 PM, nadim khemir wrote: >> Because a TODO means that it is not done not: it might happend to be >> done but >> I'm not really sure, maybe I get lucky. >> >> Either one removes the TODO and all is fine. Or it might just be a >> side effect >> that you haven't planned that makes the test pass. Calling "unexpected >> things" "features" does not make me feel more sure about quality. (I missed the above part of the original post) The red herring is "unexpected", and it's good that wording was removed from TH 3. The feature is expected, but that it worked is unexpected. You're talking like the TODO test is some slap-dash bit of code. The TODO test is crafted to test a specific feature or bug, just like any other test. What you're asking, essentially, is how do you know that the passing TODO test is testing what it's supposed to test? Well, remove "TODO" from that question and it reduces to the more general question: how do you know a passing test is testing what it's supposed to test? That is a question with plenty of answers. >> This is exactely why I complained loud about Test::Exception::dies_ok. >> It died >> but not for the problem I expected. Other may be better or have more luck >> than me though. There's nothing wrong with dies_ok(), if I understand your complaint. It does what it says on the tin, tests that the code died. And that's useful, sometimes you don't know or don't care or can't accurately predict what the exception will be. In these cases, without dies_ok() you'd have to fake up some static exception for throws_ok(). This doesn't mean that people don't use dies_ok() when they should use throws_ok(). Every tool is open to abuse. The solution is not to remove the tool, but figure out why it's being abused. Maybe the answer is as simple as putting throws_ok() first in the Test::Exception documentation? As an analogy, look at ok() vs is() in Test::More. ok() just checks that the statement is true, it doesn't care what it actually is. is() gets more specific. In general, it's better to use is() but often you don't know (or care) what the value actually is, just that it's true. Removing ok() from Test::More would increase the resolution but also remove a lot of flexibility, just as removing dies_ok() would. >> I really would like to have -Werror for my tests. A warning is "hey, you might have done something wrong!" A passing TODO test is "hey, you might have done something right!" :) -- If at first you don't succeed--you fail. -- "Portal" demo
Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?
On Dec 2, 2007, at 4:11 PM, Michael G Schwern wrote: Fergal Daly wrote: Another downside of using TODO like this is that when the external module is fixed, you have to release a new version of your module with the TODOs removed. These tests will start failing for anyone who upgrades your module but not broken one but in reality nothing has changed for that user, As long as you're releasing a new version, why would you not upgrade your module's dependency to use the version that works? Yep, that's it exactly. See, Perl::Critic's problem was that the PPI SVN repository had important bug fixes for a almost a full year before and actual PPI release was pushed to CPAN. Some of those bugs caused significant false negatives in Perl::Critic's code checks. The test was right, but the code was wrong which sure seems like a perfect definition of a TODO test. I personally was developing against the SVN PPI and adding TODOs and workarounds to support the older version. Since PPI 1.200 was released with all of the bug fixes and we released a Perl::Critic that requires 1.200, I fully agree that we *should* have removed the TODO flag on those tests. But there's always other work to be done... Patches welcome. :-) The problem with skipped tests is that they're easier for developers to ignore than TODO tests. And you have to now worry about three things instead of two: is the code right, is the test right, and is the skip conditional right. I've gotten the latter wrong before and lean toward TODO tests as a result. Chris
Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?
On Dec 2, 2007, at 1:34 PM, nadim khemir wrote: Because a TODO means that it is not done not: it might happend to be done but I'm not really sure, maybe I get lucky. Either one removes the TODO and all is fine. Or it might just be a side effect that you haven't planned that makes the test pass. Calling "unexpected things" "features" does not make me feel more sure about quality. This is exactely why I complained loud about Test::Exception::dies_ok. It died but not for the problem I expected. Other may be better or have more luck than me though. I really would like to have -Werror for my tests. Cheers, Nadim. Then how do you prefer to express functionality that might work or might not depending on what other modules the user has installed on his/her machine? I think skip() is inferior to TODO in such cases because the latter expresses the author's intention to make this feature work universally in the future. For skip(), you don't know whether it expectedly failed or unexpectedly succeeded -- it's just skipped. Chris
Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?
Fergal Daly wrote: >> As long as you're releasing a new version, why would you not upgrade your >> module's dependency to use the version that works? > > Your module either is or isn't usable with version X of Foo. > > If it is usable then you would not change your dependency before or > after the bug in version X is fixed (maybe I have a good reason not to > upgrade Foo and you wouldn't want your module to refuse to install if > it is actually usable). > > If it isn't usable then marking your tests as TODO was the wrong thing > to do in the first place, you should have bailed out due to > incompatibility with version X and not bothered to run any tests at > all. I think Extutils::MM does not have any way to specify complex > version dependencies but with Module::Build you could say ETOOBINARY Modules do not have a binary state of working or not working. They're composed of piles of (often too many) features. Code can be shippable without every single thing working. The TODO test is useful when the working version *does not yet* exist. If it's a minor feature or bug then rather than hold up the whole release waiting for someone else to fix their shit, you can mark it TODO and release. This is the author's decision to go ahead and release with a known bug. We do it all the time, just not necessarily with a formal TODO test. > I am basically against the practice of using TODO to cope with > external breakage. Not taking unexpected passes seriously encourages > this practice. Apart from there being other ways to handle external > breakage that seem easier, using TODO is actually dangerous as it can > cause false passes in 2 ways. Says version X of Foo has a non-serious > bug so you release version Y of Bar with some tests marked TODO. The > we risk Maybe we're arguing two different situations. Yours seems to be when there is a broken version of a dependency, but a known working version exists. In this case, you're right, it's better resolved with a rich dependency system. My case is when a working version of the dependency does not exist, or the last working version is so old it's more trouble than it's worth. In this case the author decides the bug is not critical, can't be worked around and doesn't want to wait for fix in the dependency. The decision is whether or not to release with a known bug. After that, wrapping it in a TODO test is just an alternative to commenting it out. Compare with the more common alternative for shipping with a known bug which is to simply not have a test at all. > 1 Version X+1 of Foo is even worse and will cause Bar to eat your dog. > Sadly for your dog, the test that might have warned him has been > marked TODO. If they release Bar with a known bug against Foo X where your dog's fur is merely a bit ruffled, then that's ok. If version X+1 of Foo causes Bar to eat your dog then why didn't their tests catch that? Was there not a "dog not eaten" test? If not then that's just an incomplete test, the TODO test has nothing to do with that. The "dog not eaten" test wouldn't have been part of the TODO test, that part worked fine when the author released and they'd have gotten the "todo passed" message and known to move it out of the TODO block. Or maybe they're just a cat person. Point is, there's multiple points where good testing practice has to break down for this situation to occur. The use of TODO test is orthogonal. > 2 You're using version X-1 of Foo, everything is sweet, your dog can > relax. You upgrade to version Y+1 of Bar which has a newly introduced > dog-eating bug. This bug goes undetected because the tests are marked > TODO. So long Fido. That's the author's (poor) decision to release with a known critical dog eating bug. The fact that it's in a TODO test is incidental. > I still have not seen an example of using TODO in this manner that > isn't better handled in a different way. > > As before, I am not advocating changing the current Test::* behaviour > to fail on unexpected passes as that would just be a mess. It's just > that whenever this is discussed it ends up with people advocating what > I consider wrong and dangerous uses of TODO and so I am pointing this > out again, Most of the cases above boil down to "author decided to release with a known critical bug" or "tests didn't check for a possible critical bug". You're right in that marking something TODO is not an excuse to release with a known critical bug, but I don't think anyone's arguing that. -- I do have a cause though. It's obscenity. I'm for it. - Tom Lehrer
Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?
If you reply to this, please make sure you reply to the 2 cases
involving the dog, this is my main objection to using TODO tests in
this manner.
On 02/12/2007, Michael G Schwern <[EMAIL PROTECTED]> wrote:
> Fergal Daly wrote:
> > One of the supposed benefits of using TODO is that you will notice
> > when the external module has been fixed. That's reasonable but I don't
> > see a need to inflict the confusion of unexpectedly passing tests on
> > all your users to achieve this.
>
> Maybe we should just change the wording and presentation so we're not
> inflicting so much.
>
> Part of the problem is it screams "OMG! UNEXPECTEDLY SUCCEEDED!" and the user
> goes "whoa, all caps" and doesn't know what to do. It's the most screamingest
> part of Test::Harness 2.
>
> Fortunately, Test::Harness 3 toned it down and made it easier to identify
> them.
>
> Test Summary Report
> ---
> /Users/schwern/tmp/todo.t (Wstat: 0 Tests: 2 Failed: 0)
> TODO passed: 1-2
>
> TAP::Parser also has a "todo_passed" test summary method so you can
> potentially customize behavior of passing todo tests at your end.
>
>
> I agree with Eric, these tests are extra credit. "Unexpectedly working
> better" != "failure" except in the most convoluted situations. Their
> intention is to act as an alternative to commenting out a test which you can't
> fix right now. An executable TODO list that tells you when you're done, so
> you don't forget.
>
> It should not halt installation, nothing's wrong as far as the user's
> concerned. However, it does mean "investigate" and it would be nice if this
> information got back to the author. It would be nice if CPAN::Reporter
> reported passing TODO tests... somehow.
>
>
> > Another downside of using TODO like this is that when the external
> > module is fixed, you have to release a new version of your module with
> > the TODOs removed. These tests will start failing for anyone who
> > upgrades your module but not broken one but in reality nothing has
> > changed for that user,
>
> As long as you're releasing a new version, why would you not upgrade your
> module's dependency to use the version that works?
Your module either is or isn't usable with version X of Foo.
If it is usable then you would not change your dependency before or
after the bug in version X is fixed (maybe I have a good reason not to
upgrade Foo and you wouldn't want your module to refuse to install if
it is actually usable).
If it isn't usable then marking your tests as TODO was the wrong thing
to do in the first place, you should have bailed out due to
incompatibility with version X and not bothered to run any tests at
all. I think Extutils::MM does not have any way to specify complex
version dependencies but with Module::Build you could say
requires => {
'Foo' => ' < X '
}
and you leave your tests untouched. When Foo X+1 comes out, you just
update your requires to say Foo => '!X' and everybody's happy.
I am basically against the practice of using TODO to cope with
external breakage. Not taking unexpected passes seriously encourages
this practice. Apart from there being other ways to handle external
breakage that seem easier, using TODO is actually dangerous as it can
cause false passes in 2 ways. Says version X of Foo has a non-serious
bug so you release version Y of Bar with some tests marked TODO. The
we risk
1 Version X+1 of Foo is even worse and will cause Bar to eat your dog.
Sadly for your dog, the test that might have warned him has been
marked TODO.
2 You're using version X-1 of Foo, everything is sweet, your dog can
relax. You upgrade to version Y+1 of Bar which has a newly introduced
dog-eating bug. This bug goes undetected because the tests are marked
TODO. So long Fido.
I still have not seen an example of using TODO in this manner that
isn't better handled in a different way.
As before, I am not advocating changing the current Test::* behaviour
to fail on unexpected passes as that would just be a mess. It's just
that whenever this is discussed it ends up with people advocating what
I consider wrong and dangerous uses of TODO and so I am pointing this
out again,
F
Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?
Fergal Daly wrote: > One of the supposed benefits of using TODO is that you will notice > when the external module has been fixed. That's reasonable but I don't > see a need to inflict the confusion of unexpectedly passing tests on > all your users to achieve this. Maybe we should just change the wording and presentation so we're not inflicting so much. Part of the problem is it screams "OMG! UNEXPECTEDLY SUCCEEDED!" and the user goes "whoa, all caps" and doesn't know what to do. It's the most screamingest part of Test::Harness 2. Fortunately, Test::Harness 3 toned it down and made it easier to identify them. Test Summary Report --- /Users/schwern/tmp/todo.t (Wstat: 0 Tests: 2 Failed: 0) TODO passed: 1-2 TAP::Parser also has a "todo_passed" test summary method so you can potentially customize behavior of passing todo tests at your end. I agree with Eric, these tests are extra credit. "Unexpectedly working better" != "failure" except in the most convoluted situations. Their intention is to act as an alternative to commenting out a test which you can't fix right now. An executable TODO list that tells you when you're done, so you don't forget. It should not halt installation, nothing's wrong as far as the user's concerned. However, it does mean "investigate" and it would be nice if this information got back to the author. It would be nice if CPAN::Reporter reported passing TODO tests... somehow. > Another downside of using TODO like this is that when the external > module is fixed, you have to release a new version of your module with > the TODOs removed. These tests will start failing for anyone who > upgrades your module but not broken one but in reality nothing has > changed for that user, As long as you're releasing a new version, why would you not upgrade your module's dependency to use the version that works? -- I am somewhat preoccupied telling the laws of physics to shut up and sit down. -- Vaarsuvius, "Order of the Stick" http://www.giantitp.com/comics/oots0107.html
Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?
http://www.mail-archive.com/perl-qa@perl.org/msg06865.html has the previous round on this topic. My memory is hazy but my view was that people are using TODO in strange ways and making this a failure would break that. The strange way I remember (and has been brought up again by Chris Dolan) is related to dealing with external modules that are broken. The idea is that you mark a test TODO when it depends on an external module whose latest version is broken. To me, this seems far better handled by checking if the external dep is working correctly or not and if not, SKIPing the affected tests. One of the supposed benefits of using TODO is that you will notice when the external module has been fixed. That's reasonable but I don't see a need to inflict the confusion of unexpectedly passing tests on all your users to achieve this. You could do that with a developer-only test and that test should also be sent to the maintainer of the broken module. Another downside of using TODO like this is that when the external module is fixed, you have to release a new version of your module with the TODOs removed. These tests will start failing for anyone who upgrades your module but not broken one but in reality nothing has changed for that user, the installed modules are still identical but the tests that were considered "ok to fail" have now morphed into "must pass". Again this is avoided by simply skipping the tests if you find the well-known breakage in the external module. So I agree with you but a lot of other people don't, F On 02/12/2007, nadim khemir <[EMAIL PROTECTED]> wrote: > The subject says it all. IE: > > All tests successful (2 subtests UNEXPECTEDLY SUCCEEDED), 7 tests skipped. > Passed TODO Stat Wstat TODOs Pass List of Passed > --- > t/20_policies.t 152 578 583 > > (nice reporting though) > > Nadim. >
Re: shouldn't "SKIPPED" mean failure?
# from Paul Johnson
>How is "didn't do what I expected" *ever* any sort of success?
It *did* do what the *test* expected ("ok(1)"). Yes, the TODO expected
it to fail, but if it is passing, you have "more than success".
It might need attention, but "failure" is a subset of "needs attention"
or "unexpected", not vice-versa.
>Just playing devil's advocate here really, but experience has taught me
>to be rather conservative when it comes to tests and that I'm not
>clever enough to predict all the things that can go wrong, so the fewer
>I let through the better.
$who lets $what through $where? Tests on the CPAN have many contexts,
so defaults need to work in all of them and any options in the harness
should do the right thing in most of them. Feel free to apply your own
local policy as far as whether to ship with passing todos or not, but
calling it "FAILED" in the harness is the wrong way to go-about that.
If it were an option to report "extra success" as "failure", I'll bet
that we'll end up with one or more cpan testers having it on
(inadvertently or not.) Then the red 'failed' lights go on for a given
platform (because it happens to work *better*) and we're left
saying "what the hell were we thinking?"
We don't report skips as failure, even though some skips actually
mean "missing features" or "unable to test in this context" -- which is
a case of "less success" than a todo passed.
Finding humor in the irony is left as an exercise for the reader.
--Eric
--
We who cut mere stones must always be envisioning cathedrals.
--Quarry worker's creed
---
http://scratchcomputing.com
---
Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?
# from nadim khemir # on Sunday 02 December 2007 11:34: >> How is "extra credit" *ever* any sort of failure? > >Because a TODO means that it is not done not: it might happend to be > done but I'm not really sure, maybe I get lucky. No, the latter is almost exactly what "todo" means. More precisely: it might pass in some situations/platforms/days-of-week, but addressing the case where it fails is an item on the todo list. ok 1 # TODO blah blah blah That looks like success. You can't treat passing TODO tests as failures. Treat them as a different sort of success, flag them as "unexpected" or "requires attention" or "wibble" or whatever, but don't mark them as failures. Ever. Is the problem simply that we have only "PASSED" and "FAILED" to choose from as answers? If so, fix the problem where the problem lies instead of trying to unintuitively wedge more information into a boolean. --Eric -- To succeed in the world, it is not enough to be stupid, you must also be well-mannered. --Voltaire --- http://scratchcomputing.com ---
Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?
On Sunday 02 December 2007 18:51, Eric Wilhelm wrote: > # from Ovid > > # on Sunday 02 December 2007 07:47: > >--- nadim khemir <[EMAIL PROTECTED]> wrote: > >> The subject says it all. IE: > > ... > > >> t/20_policies.t 152 578 583 > > > >It just means "you need to investigate this further". Personally, I > >would like to see it optionally mean failure,... > > How is "extra credit" *ever* any sort of failure? Because a TODO means that it is not done not: it might happend to be done but I'm not really sure, maybe I get lucky. Either one removes the TODO and all is fine. Or it might just be a side effect that you haven't planned that makes the test pass. Calling "unexpected things" "features" does not make me feel more sure about quality. This is exactely why I complained loud about Test::Exception::dies_ok. It died but not for the problem I expected. Other may be better or have more luck than me though. I really would like to have -Werror for my tests. Cheers, Nadim.
Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?
On Sun, Dec 02, 2007 at 09:51:49AM -0800, Eric Wilhelm wrote: > How is "extra credit" *ever* any sort of failure? How is "didn't do what I expected" *ever* any sort of success? Just playing devil's advocate here really, but experience has taught me to be rather conservative when it comes to tests and that I'm not clever enough to predict all the things that can go wrong, so the fewer I let through the better. -- Paul Johnson - [EMAIL PROTECTED] http://www.pjcj.net
Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?
On Dec 2, 2007, at 9:37 AM, nadim khemir wrote: The subject says it all. IE: All tests successful (2 subtests UNEXPECTEDLY SUCCEEDED), 7 tests skipped. Passed TODO Stat Wstat TODOs Pass List of Passed -- - t/20_policies.t 152 578 583 (nice reporting though) Nadim. No. In this case, it means that we coded against PPI v1.118 which had bugs which were fixed in PPI v1.200. So, the passing TODO tests mean that Perl::Critic is behaving correctly for the more modern PPI which you presumably have installed. In this fashion, we use TODO tests to track when bugs in our dependent packages get fixed, and then when we can remove workarounds therefore. Chris
Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?
# from Ovid # on Sunday 02 December 2007 07:47: >--- nadim khemir <[EMAIL PROTECTED]> wrote: >> The subject says it all. IE: ... >> t/20_policies.t 152 578 583 > >It just means "you need to investigate this further". Personally, I >would like to see it optionally mean failure,... How is "extra credit" *ever* any sort of failure? It does mean "the tests need to be adjusted", and maybe "the report should contain more info than just a boolean 'true'." But it should definitely not even be an option to report it as a failure. In an automated publishing tool, I would treat it as non-shippable by default but have an option to ship it anyway. For instance: a tricky cross-platform bug might fail on machines that the author doesn't have. In nightly smoke reports, you might have the "everything's ok" set to silent, but would want to send mail when TODO()s pass. Perhaps that means prove's return code is non-zero, or that the $answer ne "Result: PASS\n", but only if those are treated as "not failure". YAML? --Eric -- "Beware of bugs in the above code; I have only proved it correct, not tried it." --Donald Knuth --- http://scratchcomputing.com ---
Re: TAP::Builder
--- Ovid <[EMAIL PROTECTED]> wrote:
> Side note: those features I really want control over in
> Test::Harness
> are the plan() and ok() methods. There's no clean way for me to do
> that. Just look at the constructor:
>
> my $Test = Test::Builder->new;
> sub new {
> my($class) = shift;
> $Test ||= $class->create;
> return $Test;
> }
>
> The class name is hard-coded in there.
Note to self: don't post while hung over from the London Perl
Workshop. What I just said is rather confusing. What I *need* is a
way to easily replace Test::Builder with an appropriate subclass. I
think I can replace the builder() method in Test::Builder::Module:
sub builder {
return Test::Builder->new;
}
But it would still be nice to have something a bit more subtle than a
sledgehammer:
sub Test::Builder::Module::builder { ... }
Cheers,
Ovid
--
Buy the book - http://www.oreilly.com/catalog/perlhks/
Perl and CGI - http://users.easystreet.com/ovid/cgi_course/
Personal blog - http://publius-ovidius.livejournal.com/
Tech blog - http://use.perl.org/~Ovid/journal/
Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?
On Sunday 02 December 2007 16:47, Ovid wrote: > --- nadim khemir <[EMAIL PROTECTED]> wrote: > > The subject says it all. IE: > > > > All tests successful (2 subtests UNEXPECTEDLY SUCCEEDED), 7 tests > > skipped. > > Passed TODO Stat Wstat TODOs Pass List of Passed > > --- > > > t/20_policies.t 152 578 583 > > It just means "you need to investigate this further". Personally, I > would like to see it optionally mean failure, but you'd break a huge > number of modules on the CPAN which have passing TODO tests if you made > that mandatory. Breaking the toolchain is bad. > > Cheers, > Ovid Not when the tool chain is bad ;) But fine by me, if there was an option that I could use to force that kind of behavior on my modules, I'd be happy enough. Nadim. PS: I'll also report the passing tests to the module author.
Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?
--- nadim khemir <[EMAIL PROTECTED]> wrote: > The subject says it all. IE: > > All tests successful (2 subtests UNEXPECTEDLY SUCCEEDED), 7 tests > skipped. > Passed TODO Stat Wstat TODOs Pass List of Passed > --- > t/20_policies.t 152 578 583 It just means "you need to investigate this further". Personally, I would like to see it optionally mean failure, but you'd break a huge number of modules on the CPAN which have passing TODO tests if you made that mandatory. Breaking the toolchain is bad. Cheers, Ovid -- Buy the book - http://www.oreilly.com/catalog/perlhks/ Perl and CGI - http://users.easystreet.com/ovid/cgi_course/ Personal blog - http://publius-ovidius.livejournal.com/ Tech blog - http://use.perl.org/~Ovid/journal/
Re: TAP::Builder
--- Michael G Schwern <[EMAIL PROTECTED]> wrote:
> This is easier to do with Test::Builder because custom features meant
> custom
> functions. Custom functions avoid functionality overlap and users
> write code
> specifically for them. Test::Harness and prove have no such luxury.
Right, let's think about this. Test::Harness features tend to impact
an *entire* test suite and that can be a hard problem to solve. What
does it mean if someone says they want a GUI and colored test results?
Those aren't really the same thing and they could very well try to hook
into the same functions. If you can write code which intelligently
decides the *intent* of potentially conflicting plugins, you'll have
code which can pass the Turing test. We've had several false starts on
this and we decided to focus on those things we can really solve and
understand. When people start writing more and more stuff around
Test::Harness, then we might have a better understanding of what the
real world needs are.
Further, many people have many different needs for a test suite.
Test::Builder essentially gets to limit itself to printing "yes" or
"no", over and over again. So far it's been a one-size-fits-all suit.
Sure, you can write your own testing module, but you're still just
printing "yes" or "no". I don't think it's really fair to take the new
Test::Harness to task for not having immediately solved a far more
difficult problem.
Side note: those features I really want control over in Test::Harness
are the plan() and ok() methods. There's no clean way for me to do
that. Just look at the constructor:
my $Test = Test::Builder->new;
sub new {
my($class) = shift;
$Test ||= $class->create;
return $Test;
}
The class name is hard-coded in there. And look at how much the plan()
method does! Sure, it's all logically related to the plan, but if I
want to customize it, I potentially have to duplicate an awful lot of
code (which I had to do with Test::Aggregate).
> How do you do this without balkanizing TAP::Parser into a bazillion
> mutually
> exclusive subclasses? Something like App::ProvePlus is right out,
> it's just shifting the dumping ground.
I suspect we need a minimal subset of traits (Class::Trait::Lite?)
which can be bundled with Test::Harness.
Cheers,
Ovid
--
Buy the book - http://www.oreilly.com/catalog/perlhks/
Perl and CGI - http://users.easystreet.com/ovid/cgi_course/
Personal blog - http://publius-ovidius.livejournal.com/
Tech blog - http://use.perl.org/~Ovid/journal/
shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?
The subject says it all. IE: All tests successful (2 subtests UNEXPECTEDLY SUCCEEDED), 7 tests skipped. Passed TODO Stat Wstat TODOs Pass List of Passed --- t/20_policies.t 152 578 583 (nice reporting though) Nadim.
