Re: Why are we adding more kwalitee tests?
--- Andy Lester [EMAIL PROTECTED] wrote: Why are we worrying about these automated kwalitee tests? What will happen once we find that DBIx::Wango has only passed 7 of these 23 items on the checklist? I am not the one to answer this, but I'm curious to know where you are coming from. Is it: 1) Metrics, in general, are often misused and do more harm than good. 2) The Module-CPANTS kwalitee metrics are poorly chosen. 3) The Module-CPANTS kwalitee metrics are an unwelcome distraction to CPAN authors who should be focusing on improving their module quality, not on improving their Module-CPANTS kwalitee score. 4) Schwern's original kwalitee vision in http://www.nntp.perl.org/group/perl.qa/149 is not worth worrying about. Or all of the above? I have sympathy for 1) above because I've seen first hand the dysfunctional behaviour that can flow from people focusing on improving their metrics at the expense of doing a good job. For example, a support group whose performance was measured by the number of calls they closed per month, so they started to close customer calls and reopen the same issue with a new call (which annoyed the customer, but, hey, they improved their performance). Many other examples could be given. See also Brian Marick's How to misuse Code Coverage paper. To clarify point 2) above do you equally feel we shouldn't worry about the CPAN top 100 code coverage metrics published by Paul Johnson? Personally, I see the publication of CPAN code coverage and kwalitee metrics as useful data for CPAN authors to help them improve their offerings and catch blunders. Of course, they shouldn't become a distraction from the primary task of producing a quality module, i.e. one with great user interface, doco, performance, robustness, maintainability, portability, error handling, extensibility, ... /-\ Do you Yahoo!? The New Yahoo! Movies: Check out the Latest Trailers, Premiere Photos and full Actor Database. http://au.movies.yahoo.com
Re: Adding more kwalitee tests
Apart from a user wanting to know which version of perl a module works with, it's a general quality (not kwalitee;-) principle to first specify, then test that the specification is met. So I would say that specifying which versions of perl your module is meant to work with should be done before testing it against those versions. In addition to META.yml, your module's version requirements should be enforced programmatically. There are ways and means to test that as well :) Well, Damian says so. :-) (p.405 of Perl Best Practices: Enforce your version requirements programmatically). Another possible kwalitee check: the perl version in META.yml should match the version enforced programmatically. BTW, is there any plan to mine PBP for possible new kwalitee tests? Well, not immediately here, but there is Perl::BestPractices being slowly pulled together which all the necesary work, and so I imagine you could plug the two together relatively easily somewhere down the track. Adam K
Bug report
Dear all,
I have submitted a bug report to rt.cpan.org and have been told to
forward a copy of the bug report on to this list. The bug report is
attached to this email.
FYI - the use Test::MockObject line causes the code to die because
Test::MockObject now depends on UNIVERSAL::can (the *module* - not the
method). This module is where the problem lies (and is also where I've
filed the bug on rt.cpan.org). I hope this kind of explains the problem!
I am not a subscriber to this list, so please contact me directly if you
have any questions.
Hope this helps,
--
Dan Rowles
Outcome Technologies
_
t: +44 (0)207 656 2460
f: +44 (0)709 230 6588
m: +44 (0)798 076 8143
e: [EMAIL PROTECTED]
w: http://www.outcometechnologies.com
BUPA House
15-19 Bloomsbury Way
London WC1A 2BA
_
***This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they are
addressed. If you receive this message in error, please return it to the
sender.***
---BeginMessage---
Greetings,
This message has been automatically generated in response to your bug
report about UNIVERSAL-can, a summary of which appears below.
There is no need to reply to this message right now. Your bug in UNIVERSAL-can
has been assigned an ID of [cpan #14477]. Please include the string:
[cpan #14477]
in the subject line of all future correspondence about this issue. To do so,
you may reply to this message.
Thank you,
[EMAIL PROTECTED]
-
The following code dies on perl 5.8.3:-
#! perl
use strict;
use warnings;
# UNCOMMENT THIS LINE TO SEE THIS CODE BREAK
#use Test::MockObject;
if(UNIVERSAL::isa( = ARRAY)) {
print YES\n;
}
else {
print NO\n;
}
---End Message---
Re: Why are we adding more kwalitee tests?
Andy Lester wrote: Why are we worrying about these automated kwalitee tests? What will happen once we find that DBIx::Wango has only passed 7 of these 23 items on the checklist? I thought the same thing, until kwalitee turned competative. Now we have a situation in which a large number of authors have altered their behaviour to match the scoring mechanism. So once you find out DBIx::Wango only passed 7 out of 23, it will go into the author's average, and if he ever looks presumably the competative spirit will kick in and he's fix some of the problems Adam K
Re: Why are we adding more kwalitee tests?
On 6 Sep 2005, at 08:10, Adam Kennedy wrote: So once you find out DBIx::Wango only passed 7 out of 23, it will go into the author's average, and if he ever looks presumably the competative spirit will kick in and he's fix some of the problems That's assuming that everyone is competitive. I'm not, and so I'm not about to crank out 62 new releases just because my distributions aren't up with the current fashions in cargo-culted tests. -- Richard Clamp [EMAIL PROTECTED]
Re: Why are we adding more kwalitee tests?
On Tue, Sep 06, 2005 at 05:10:40PM +1000, Adam Kennedy ([EMAIL PROTECTED]) wrote: So once you find out DBIx::Wango only passed 7 out of 23, it will go into the author's average, and if he ever looks presumably the competative spirit will kick in and he's fix some of the problems But will the author actually care? Will the author even know this exists? Are you going to send email to Bob and say Hey, Bob, you only passed 7 of 23 things? What's Bob going to say in return? I see a couple of options: * Nothing, not even seeing the email as interesting. * Hey, that's pretty cool, I'm going to change my code! * Yeah, so what? * Fuck you, you high-and-mighty assholes of correctness! I put something on CPAN for people to use and you jerks come along and tell me that I don't meet your oh-so-high standards! I would SERIOUSLY recommend that before anything goes further that people look at the end result of what they're trying to acheive. I'm suggesting that you actually draft the email, or mock up the web page, or whatever it may be, communicating the failings of people and their code. I understand fully that nobody's intent is to tell people that they suck, but there's a non-zero chance, and probably closer to 50%, that people will see it that way. xoxo, Andy -- Andy Lester = [EMAIL PROTECTED] = www.petdance.com = AIM:petdance
Re: Why are we adding more kwalitee tests?
On Tue, Sep 06, 2005 at 09:12:43AM -0400, Christopher H. Laco ([EMAIL PROTECTED]) wrote: If it serves no purpose for you, ignore it and go on with life; as apposed to spending email list cycles on a CPANTS-is-bad-why-are-we-doing-this diatribe. It's not as simple as just ignore it if the result of your actions are that people stop uploading to CPAN, or new authors are steered away, for fear of scorn and ridicule. -- Andy Lester = [EMAIL PROTECTED] = www.petdance.com = AIM:petdance
Re: Adding more kwalitee tests
Ivan Tubert-Brohman wrote:
Sorry for my ignorance, but I had never even heard of this option. I
don't find any way of setting it via MakeMaker or Module::Build. Do I
have to edit META.yml by hand? Would it get overritten by 'make dist'?
The spec is here:
http://module-build.sourceforge.net/META-spec-v1.2.html
It can be added with Module::Build with the meta_add option. E.g., as
part of the arguments to new() in your Build.PL:
meta_add= {
noindex = { directory = [ qw/examples/ ] }
},
It would be nice if Module::Build added a noindex option directly. Maybe
after the 0.27 release.
Regards,
David Golden
Re: Why are we adding more kwalitee tests?
Andy Lester wrote: On Tue, Sep 06, 2005 at 09:12:43AM -0400, Christopher H. Laco ([EMAIL PROTECTED]) wrote: If it serves no purpose for you, ignore it and go on with life; as apposed to spending email list cycles on a CPANTS-is-bad-why-are-we-doing-this diatribe. It's not as simple as just ignore it if the result of your actions are that people stop uploading to CPAN, or new authors are steered away, for fear of scorn and ridicule. Why would they stop uploading? How would they, the new uploaders, even know about CPANTS? It's not like uploaded files automatically return a scathing email and an html response page that says your module sucks; failed CPANTS kwalatee. Go away. This is no different than CPAN Testers. I can upload dists till the cows come home. If my module failes every single cpan testers report, who cares? That doesn't stop or disuade people from uploading does it? Of course not, so why would CPANTS kwalitee be any different? -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: Why are we adding more kwalitee tests?
Andy Lester wrote: Why are we worrying about these automated kwalitee tests? What will happen once we find that DBIx::Wango has only passed 7 of these 23 items on the checklist? I don't have any problem with someone proposing or running these kinds of automated test. It's helpful feedback to me as a distribution author. Whether I choose to do something about the feedback or not is something else. Viewed as a tool for authors, automated kwalitee testing has value. As a tool for module users, it's not so clear cut. I would be concerned if CPANTS Kwalitee showed up on the CPAN page for a distribution. That would make it official and judgmental I don't think that an arbitrary and controversial (at least in part) metric should be so enshrined. But I think the collection of metrics themselves is value-neutral. If someone wants to write the tests and invest processor cycles on it, more power to them. Regards, David Golden
Re: Adding more kwalitee tests
Adam Kennedy wrote: missing_no_index: Only the libraries (and .pod docs) in your dist should be indexed) things in inc/ and t/ and examples/ etc should NOT be indexed. Thus, there should be a no_index entry in the meta.yml for all of these directories (if they have .pm files in them). I like it. (N.B. Since t/ is not indexed already and many people write test support modules, I'd recommend against needing those to be explictly included in a no_index) has_perl_dependency: My initial thought was great! Then I realized that encouraging people to stick one one in may be worse than having none at all as quick (but incorrect) information may be less desireable than no information. --- bad_perl_dependency: The caching and mass processing facilities in PPI are getting close to being able to regularly process all of CPAN. The first demo of this I'm hoping to do is a use of Perl::MinimumVersion to find cases where a module is listed as working with (for example) Perl 5.005, but the module syntax shows that it needs 5.006. (and thus the version dependency is wrong). I later plan to expand this to include cases where it says it should run with 5.005, but it has a dependency on some other module (recursively) that needs 5.006, or you accidentally put 5.006 code in your test scripts, and thus fails depsite the code in that actual module for the module being ok. I like this a lot. I'd drop has_perl_dependency and just have bad_perl_dependency fail if there is no perl dependency listed. I.e., assume that no perl dependency is equivalent to use perl 0 or maybe some arbitrarily old version of perl 5 (e.g. 5.004). Regards, David Golden
Re: Why are we adding more kwalitee tests?
On Tue, Sep 06, 2005 at 10:07:02AM -0400, Christopher H. Laco ([EMAIL PROTECTED]) wrote: Why would they stop uploading? How would they, the new uploaders, even know about CPANTS? It's not like uploaded files automatically return a scathing email and an html response page that says your module sucks; failed CPANTS kwalatee. Go away. I don't know. That's why I ASKED THE QUESTION of what will be done with the information about their results. -- Andy Lester = [EMAIL PROTECTED] = www.petdance.com = AIM:petdance
Re: Why are we adding more kwalitee tests?
-BEGIN PGP SIGNED MESSAGE- Moin, On Tuesday 06 September 2005 09:10, Adam Kennedy wrote: Andy Lester wrote: Why are we worrying about these automated kwalitee tests? What will happen once we find that DBIx::Wango has only passed 7 of these 23 items on the checklist? I thought the same thing, until kwalitee turned competative. Now we have a situation in which a large number of authors have altered their behaviour to match the scoring mechanism. So once you find out DBIx::Wango only passed 7 out of 23, it will go into the author's average, and if he ever looks presumably the competative spirit will kick in and he's fix some of the problems I thought that was the whole point behind this. And I look forward to learn about problems I didn't even know existed before. Of course, it is up to me (Joe Author) to weed out virtual from real problems. (Given the low test-coverage most modules have, I'd say most people will ignore CPANTS, too). It is a tool, you are free to not use it. :) best wishes, Tels - -- Signed on Tue Sep 6 18:03:22 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. I am soo clumsy today. *crash* -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iQEVAwUBQx3BCHcLPEOTuEwVAQHrwAf/TPMXDtH6/p/rq6GwQEqzqePvN/MHbnB6 qaODl09NOHPVXQv74jZWUF4L9/2Dq5/RPYEUiZVbqaIQZEpQ9ChTY2ZeYLXhC5ON v1/hBwuS/tVmlI1JtMiGuNfrk5WAPmEkLeiPM7oB09EvReKQ9WAwyFnncgD4ap+r cwTkR6pld7wKje4nm6uIwuxu84XBlGtz4MlzOnyxea7zYq4TZq4wRltCqxarvVIZ gWdMDmABt0RWfbR8nTEB3NP5+i4Cyt9BRwMWx5eQwcCzYnz0GI6jHNMPqXsluYG6 B1h+24IoEoJiY0NcKAwFAXeCdOLy2+Cz0B20qk48PSFXbuB3AauTxA== =SmzH -END PGP SIGNATURE-
Re: Adding more kwalitee tests
On Mon, 5 Sep 2005, Thomas Klausner wrote: has_perl_dependency: In the META.yml (assuming it exists) there is a dependency on the version of perl required to install the dist. The goal of this is to make life a little easier on installers and CPAN testers and a few other things. Many many modules should pass this test already. The main goal however, is to help enable the NEXT kwalitee test to have better coverage. Hmm. I never added this of any to my dists. It might also be annoying for people if authors add a newer version of Perl as a prereq than they have installed and some installer then goes off to fetch install a new version of Perl. (And it's a lot of hassle to find the exact lowest version of Perl you need). I'm pretty sure that CPAN and CPANPLUS will explicitly _not_ do this. The perl dependency is handled specially. -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: Why are we adding more kwalitee tests?
On Tue, 2005-09-06 at 09:12 -0400, Christopher H. Laco wrote: I really don't get why the people (not specifically you) who don't agree with, don't care for, don't care about CPANTs or more CPANTS tests spend all this effort going off on why it's such a bad thing and why it shouldn't be done. I care because *some* of the metrics don't completely suck. It'd be nice to get rid of (or improve) the useless ones while failing to add metrics such as Uses no hard tabs anywhere in the distribution, Uses BSD style formatting, Has twice as much documentation as code, Has no subroutine over 20 lines in length, Uses good variable names, Doesn't reinvent the wheel, and Runs test coverage automatically using Acme::Test::Coverage::Everyone::Everywhere. -- c
Re: Why are we adding more kwalitee tests?
--- Andy Lester wrote: But will the author actually care? Will the author even know this exists? Are you going to send email to Bob and say Hey, Bob, you only passed 7 of 23 things? What's Bob going to say in return? I see a couple of options: Ah, now I see where you are coming from. I would say we should definitely not spam CPAN authors with kwalitee metric emails; that is guaranteed to really annoy a lot of them. I feel pushily promoting kwalitee metrics will do more harm than good. And I agree with chromatic that officially endorsing any particular kwalitee metric is a mistake that is likely to cause unproductive flame wars. Best, at least for now, is to simply publish some kwalitee metrics as an optional aid to enthusiastic CPAN authors. If they prove good and useful, they will naturally become better known. /-\ Do you Yahoo!? The New Yahoo! Movies: Check out the Latest Trailers, Premiere Photos and full Actor Database. http://au.movies.yahoo.com
Re: Why are we adding more kwalitee tests?
Hi!
On Wed, Sep 07, 2005 at 05:31:37AM +1000, Andrew Savige wrote:
flame wars. Best, at least for now, is to simply publish some
kwalitee metrics as an optional aid to enthusiastic CPAN authors.
If they prove good and useful, they will naturally become better known.
Some of my recent thoughts about CPANTS:
*) The metrics should help the average Perl programmer to generate better
distributions. If a few people use better techniques (that might be too
tough for 'average' programmers) it might happen that they do not get all
the points. (cf has_test_pod_coverage vs using Dever::Cover et.al)
But it's better for CPAN as a whole: A few people add pod-coverage tests
because they get a few more points.
*) CPANTS is not describing the one and only way how to write Perl / pack
distribution. It's more of an online mutliplayer game where people submit
their dists which than fight against my evil metrics.
Please do not take CPANTS too serious. It might evolve into something really
usefull, but currently it's way to simple-minded. As it says on the new
Catalyst-powered web frontend: http://cpants.perl.org
ALPHA VERSION - Hic sunt dracones!
--
#!/usr/bin/perl http://domm.zsi.at
for(ref bless{},just'another'perl'hacker){s-:+-$-gprint$_.$/}
Re: Why are we adding more kwalitee tests?
On Wed, 2005-09-07 at 05:31 +1000, Andrew Savige wrote: I feel pushily promoting kwalitee metrics will do more harm than good. And I agree with chromatic that officially endorsing any particular kwalitee metric is a mistake that is likely to cause unproductive flame wars. It's not that endorsing metrics is a problem, it's that certain metrics purport to measure things that I don't think you can (or should) measure. Maybe the problem is that CPANTS as it exists now measures some metrics better measured on the developer side, not the installer side. It's handy to run the POD coverage tests as the developer of a module, but it's not that interesting for the person installing the module to run them. It would be handy to check that the version numbers and MANIFEST match as the developer, but not really for the person installing the module. A test for forgetting to check in new files to the repository (or add them to the MANIFEST) would be very useful as a developer, but useless to the installer. I realize that the public side of CPANTS (the existing side) can't reliably show these scores (or people could edit their META.yml files to fudge their scores), but the benefits are: - usable without uploading a module - easier for first-time module writers - works with in-house modules never intended for public distribution - provides a good final out-the-door sanity check for CPAN authors - keeps useful tests, but pushes them to where they're most useful Some kwalitee metrics are useful in both places and that's fine. I just wonder if some of PANTS should be more private. -- c
