On Sat, Apr 02, 2005 at 11:29:17AM +0000, Smylers wrote: > Indeed, and those also have odd effects: rather than being pure measures > of users' abilities/reputations/whatever, their very existence changes > how some users behave, where they do things specifically to increase XP > rather than because they have intrinsic value.
Yes, and this is the danger of any system which attempts to quantify "quality". The runaway systems you've described can result from not having enough humans in the loop checking how closely the measurement is actually jiving with reality. Its as simple as having some reliable humans rating some pathological cases, some modules which are universally agreed to have a high quality and some which are universally agreed to have a low quality, and see if their kwalitee ratings match. If they don't then corrections need to be made. You can do the same for authors. Acme::Raise_my_kwalitee is a perfect example. A low quality module with a high Kwalitee rating. One can also measure the heuristics for bias by checking to see if its just a small clump of authors which pass that test. I suspect the Test::Pod check will fall into that category. Finally, the scoreboard does have a purpose. Part of the original idea of CPANTS was to provide an automated checklist for a good distribution. Has a README... check Declares a $VERSION... check Well behaved tarball... no I know a lot of my modules don't bother with a lot of things they should. CPANTS shoves this in my face. Then, if this were a web page, the author could just click on that to get an explaination of why this is a Good Thing and what they can do to fix it. This is information which is kind of floating around out there in the collective knowledge of the community but is rarely written down [1] and when it is its often well out of date with current practice. There were issues of how to diseminate this information to authors. Many expressed "I don't want to get spam telling me some arbitrary standard says my module is wrong" so it couldn't be push, it has to be pull. How do you get authors to actually look at the CPANTS information and make corrections? Well, we like competition. Make it a game! [1] The spirit of Sam Tregar has knocked on my laptop and asked me to tell you that his "Writing Perl Modules for CPAN" is now available for free. http://www.apress.com/free/
