On 7/4/07, Barbie <[EMAIL PROTECTED]> wrote:
Hi David,
> - UNKNOWN results when prereqs are missing are downgraded to NA (just
> like FAIL reports with missing prereqs already were)
This is wrong. If a distribution has prereqs missing from
Makefile.PL/Build.PL then it should receive a FAIL report. If the tool
chain fails to install the prereq, then the report should be abandoned
and not sent at all.
I don't think it's wrong -- but we are using the phrase "prereqs are
missing" differently. CPAN::Reporter checks to see if any of the
prereqs specified in the Makefile.PL or Build.PL are missing (i.e. not
installed). This can happen if the tests were run anyway despite
missing prereqs (e.g. "force test"). In such cases, CPAN::Reporter
"downgrades" the detected test grade from FAIL to NA -- indicating
that the report results are meaningless. Since NA reports by default
aren't sent to the list or the author, it also avoids sending out such
useless results.
In 0.45, UNKNOWN results also benefit from this downgrade behavior.
CPAN::Reporter falls back to assigning an UNKNOWN grade if it can't
otherwise determine the result from parsing Test::Harness output
strings (or test.pl exit codes). These results do get sent out,
unlike NA reports, and -- as Ask demonstrated -- are generally
annoying enough to authors to cause them to complain, which helps us
find cases that we didn't previously know how to handle. (So thank
you, Ask, for taking the time to complain!)
NA is reserved for when the distribution doesn't work on a particular
version of Perl or platform. Adding broken prereqs to the mix is
confusing and misleading.
I would disagree on the first part and suggest that the definition be
broadened to include any condition where the test result should just
be ignored. Plenty of tests get run when they shouldn't -- NA should
be the grade for all of those cases.
Additionally, there is no well-defined way that I've ever seen
documented for a distribution to specify what platform it runs on or
to gracefully bail out if it's on the wrong platform. Some people
avoid creating a Makefile, which may work or may get picked up as a
Makefile.PL failure. Some people bail out of test scripts (or just
let tests fail). Detecting a "platform" NA is thus a poorly defined
heuristic.
David
