* chromatic <[EMAIL PROTECTED]> [2008-06-25 19:25]:
> Removing the game score completely would fix a lot of what I
> consider wrong with CPANTS.

I think the game is actually an excellent idea. The problem is
with the metrics. Here are some metrics that are inarguably
good:

• has_buildtool
• extracts_nicely
• metayml_conforms_to_known_spec
• no_pod_errors
• has_tests_in_t_dir
• is_prereq

Here are some that have proven harmful:

• has_test_pod
• has_test_pod_coverage

What’s the difference? It’s whether the things that are measured
are used as a proxy for some other ostensibly desirable property.

Kwalitee is helpful when it measures good form directly and
unhelpful when it measures an arbitrarily chosen indicator of one
of many ways to adhere to good form. (Good style, in contrast,
cannot be measure. Kwalitee is not quality.)

The game merely amplifies this: helpful metrics become actively
helpful and unhelpful metrics become actively unhelpful.

I am in favour of keeping the good effects of the game and
removing the bad ones. That means cleaning up the metrics roster
– for the game. Some not universally applicable metrics may still
be of interest, after all. They can stay – as long as they are
removed from the game. And by this I mean removing them entirely,
that is, they should not figure into the score at all, not even
as uncounted bonus points or such, nor even shown on any page
that ranks authors.

Take Paul Fenwick’s recent proposal in this light: he wants to
know whether a module accidentally untaints values or not, but
what he proposed is to measure whether a module does `use re
'taint';` – nevermind that this is inapplicable to a large class
of modules OT1H and doesn’t even cover all the ground OTO.
(Iterating a hash’s keys returns strings that are never tained.)
Thus it might be useful to see a page listing the distributions
whose modules do `use re 'taint';`, but it’s less than helpful
to rank module authors by how many of their modules do so.

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>

Reply via email to