# from David Golden # on Thursday 10 July 2008 10:57: >(That said, I'm not sure I'd suggest holding up a release waiting for >me/others to get it all written in the next couple weeks.)
How long has it been since the last release? ;-) >> Especially handy would be some way to characterize the failures >> which share a root cause. Perhaps a nightly aggregation/report run >> could be posted to the list? > >That's a hard problem right now. CPAN Testers 2.0 (still in the >design stage) might well support capturing TAP archives, allowing more >detailed analysis of exactly which tests fail, but that still a long >ways off. We don't really need to capture the TAP archive to know whether Module::Build is broken. The failure case we need to catch is that a dist built/installed fine with the stable M::B and failed with the alpha. This is a quite different test scheme vs what's currently happenning with most smokers. The characteristics of import are then the OS, perl version, version numbers of various related modules, CPAN(PLUS) config, and possibly some details about the filesystem layout or what-not. Which distribution being built is not so important as long as it was known-good with the stable M::B (or known to fail due to a bug which we fixed -- we actually do that sometimes.) So, e.g. a cross-section of multiple platforms/perls building File::Fu would be a basically useful check on non-XS functionality. Some dists use a compatibilly Makefile.PL (with or without passthrough) and some write that by hand, etc. Actually, someone could fabricate some fake distributions (TestMe::MB, TestMe::MBCompat, TestMe::XS, etc.) Of course, real authors are going to create failure cases we didn't think of. In short, we'll never be able to be sure that we didn't break something because there is a lot of behavior which comes from platforms and other external factors. We can only try really hard to discover in alpha what we *did* break. The current situation is that we won't be making that discovery until too late. So, help finding and fixing these sorts of bugs would go a long way toward getting regular stable releases. Thanks, Eric -- Turns out the optimal technique is to put it in reverse and gun it. --Steven Squyres (on challenges in interplanetary robot navigation) --------------------------------------------------- http://scratchcomputing.com ---------------------------------------------------