On Dec 22, 2007 8:26 PM, chromatic <[EMAIL PROTECTED]> wrote: > CPAN Testers reporting failures in every module they test and not stopping to > ask "Hey, is it possible that not everything else in the world is broken?" is > *not* an example of CPAN Testers working as expected. > > Environments where it's impossible even to *build* Perl modules are unsuitable > for smoketest reporting, as they don't provide any useful information and > they make true failures much more difficult to see and believe.
Environments where it's possible to install a CPAN Testers tool like CPAN::Reporter is -- pretty obviously -- an environment where modules can build and where there is some relevance to smoke test results. Here is Cantrell's CPAN Deps for CPAN::Reporter: http://tinyurl.com/23yd55 It's a non-trivial dependency chain. CPAN::Reporter itself tests 47 different simple test distributions to stress different grades from different paths through Module::Build and ExtUtils::MakeMaker. Here is CPAN Deps for CPAN::YACSmoke: http://tinyurl.com/243vlw Plus, YACSmoke requires CPANPLUS which has been hard to build on many systems that *do* function well. (Hopefully improved by including it in core 5.10.) These are not simple modules. Assuming that some smoker can get these to successfully send you a report, then which scenario seems more likely: (a) that the smoker's installation is fundamentally broken (b) that your distribution isn't sufficiently portable to handle their installatiion In the case of (a), if you send me actual reports that you take issue with, and if the nature of the fundamental breakage can be determined in an automated way, I'll consider adding logic to CPAN::Reporter to screen those out. And in the case of (b), there are (now) plenty of modules to help you check configurations (Devel::CheckLib, Devel::AsssertOS) or test under more unusual circumstances (Devel::Hide, Test::NoXS). I'm not saying you have to care enough about FAIL reports to go to those lengths. Feel free to ignore them -- it's just data. But if you care enough to want to avoid any FAIL reports, don't just blame the testers when you could be doing something about it instead. Regards, David
