On 10/6/07, David Cantrell <[EMAIL PROTECTED]> wrote:
> On Sat, Oct 06, 2007 at 04:59:21PM -0400, David Golden wrote:
> > (b) plenty of modules test warnings by sending stuff to STDERR.
> > (Probably shouldn't but they do.)
>
> They not only shouldn't, but simply printing to STDERR to test something
> to do with warnings is a bug. To test a warning, you need to warn() and
> capture it with $SIG{__WARN__}.
I phrased that poorly. Plenty of modules test behaviors that issue
warnings or otherwise print to STDERR without capturing STDERR before
it gets into the output stream of Test::Harness.
This is just the nature of trying to work with TAP in-band with all
other console output. Thus, the place for a new "grade" like this to
be determined is with the TAPX/Test-Harness 3.0 crowd. E.g. a harness
switch that detects warnings and causes a "RESULT:PASS/warnings
detected" result that CPAN::Reporter could parse. I just don't think
that the CPAN Testers stack should be adding any more heuristic test
distinctions that aren't reflected in the underlying test framework.
In my opinion, we're just trying to automate the capture of the output
and results of "make test" -- the same way an end user would see it.
I wouldn't want to introduce a bunch of process and IO handle
manipulations that weaken or break that connection.
David
