Let's step back a moment.
Does anyone object that CPANTS Kwalitee looks for tests? Why not apply the same arguments against has_test_* to test themselves? What if I, as a developer, choose to run test as part of my development but don't ship them. Why should I make users have to spent time waiting for my test suite to run?
Keeping in mind that this is a thought exercise, not a real argument, here are some possible reasons (and counter arguments) for including test files in a distribution and for Kwalitee to include the existence of tests
* Shipping tests is a hint that a developer at least thought about testing. Counter: It's no guarantee of the quality of testing and can be easily spoofed to raise quality.
* Tests evaluate the success of the distribution against its design goals given a user's unique system and Perl configuration. Counter: developers should take responsibility for ensuring portability instead of hoping it works unti some user breaks it.
The first point extends very nicely to both has_test_* and coverage testing. Including a test for pod/pod-coverage shows that the developer thought about it. It doesn't mean that a developer couldn't do those things and just not create a *.t file for them, of course, or create a *.t file for them and not do those things, either. The presence of a test is just a sign -- and one that doesn't require code to be run to determine Kwalitee. The flip side, of course, is that by including test that are necessary for CPANTS, a developer inflicts them on everyone who uses the code. That isn't so terrible for pod and pod coverage testing, but it's a much bigger hit for Devel::Cover.
Why not find a way to include them in the META.yml file and have the build tools keep track of whether pod/pod-coverage/code-coverage was run? Self reported statistics are easy to fake, but so are the has_test_* Kwalitee checks as many people have pointed out. Anyone who is obsessed about Kwality scores is going to fake other other checks, too. And that way, people who have customized their environments can report that they are doing it.
As to the benefits of having Devel::Cover run on many environments and recording the output, rather than suggest developers put it in a *.t file -- which forces all users to cope with it -- instead why not build it into CPANPLUS as an option along the lines of how test reporting is done. Make it a user choice, not a mandated action.
Ironically, for all the skeptical comments about "why a scoreboard" -- the fact that many people care about the Kwalitee metric suggests that it does serve some inspirational purpose.
Regards, David Golden
