For background on this email, see the following entry in my journal.
http://use.perl.org/~Alias/journal/33893
Lately, I've noticed a series of modules that are cargo-culting the use
of test_pod and test_pod_coverage in their tests.
Not only are we seeing spurious failures for users, but worse some
authors are putting Test::Pod and Test::Pod::Coverage as dependencies,
and in extreme cases, Perl::Critic as well.
Now, one argument is that it's the author's fault for not appropriately
skipping these tests if the Test::Pod module is not available.
However, in most of these cases the author would not have made these
errors AT ALL if not for the fact that Kwalitee says they should have
these tests.
The CPANTS Game just makes this pressure worse, by making it competive.
That said, I do see the value of these tests EXISTING in the
distribution, as their provide a guarentee that any new maintainers for
the modules will continue to comply with the principles those tests
implement.
BUT for most of these tests, it is BAD that the end-user EVER run them.
Thus, I would like to propose the following.
1. That the running of POD-related tests by end users is considered harmful.
2. That the test_pod and test_pod_coverage tests by modified, such that
these tests check for the mentioning of $ENV{AUTOMATED_TESTING} in the
tests scripts, ensuring that the tests not only exist, but exist in a
form that is otherwise benign for end users.
This change to the tests is targetted, in the sense that people who do
NOT have the test_pod and test_pod_coverage tests will not be further
penalized, while the people that are doing cargo-culted damaging test
scripts are the ones most likely to care about the CPANTS game point loss.
Thoughts? Alternative?
Adam K
