-----BEGIN PGP SIGNED MESSAGE----- Moin,
On Monday 14 November 2005 18:21, Chris Dolan wrote: > Hello all, > > I've just published an article about public vs. private regression > tests. I've defined private tests as t/*.t files that are for the > author only and don't go in MANIFEST. Naturally, those don't get as > much publicity as tests included in CPAN distributions. > > In the article I advocate that some tests should be private. In > particular, > 1) those that test non-critical aspects of a module (like > documentation and coding style) > 2) those that are too expensive to run often > 3) those that require special software or customization > In my conclusion I describe a possible system where authors publish > the results of private tests with their distributions as a trust- > based kwalitee system. That is, authors assert kwalitee rather than > be judged for it. > > http://www.chrisdolan.net/talk/index.php/2005/11/14/private- > regression-tests/ > > Both positive and negative feedback is very welcome! Private tests will only be run by the author, meaning they will be only run on a very small subset of all systems the modules can be used on. This limits their usefullness quite a bit. Case ein point: I can test my modules on linux, 32 bit, unthreaded, under unicode, and under perl 5.8.x. Thats about it, everything else gets really really complicated for me to set up and maintain/test. So, no win32, no mac, now solaris, no irix, no perl 5.6.x, no iso-something, no EBDIC (or however it is spelled), no threading, no 64 bit, no SMP system. As for 1), these things should matter (the "broken window analogy") and you would be surprised to know how these tests can pass on your system, and still fail on other systesm (forget to include the .pod file in MANIFEST is the most obvious one). As for 2), random testing should be employed (Math::BigInt does this, it runs 256 or so tests with random number patterns (and thus known results like "2 * A - A == A"). The tests are quite fast, but they cover only a small subset of potential values. However, since each system and user runs a new, different random set, you end up with a really huge testing number being run. (Yes, this has found some bugs) And for 3), this might be the only point I can think that private tests are usefull (I have a private testset for Graph::Easy that I use from time to time, it is not public mainly because it fails/hangs/takes forever and is work-in-progress). However, I have to actually read your article to find out what your proposal solves (compared to me just running thetest once in a while :) Hope that was usefull :) Best wishes, Tels - -- Signed on Tue Nov 15 11:04:21 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. "Now, _you_ behave!" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iQEVAwUBQ3m0pHcLPEOTuEwVAQF6lQf8CtubDMQjLdCBcEGNczxZj2Y1kTVhOU7Z bvweeJe4RWFfmd2JMw7dwiu3Sjb57hNlVkl4LwN+7vx3tm3DsnhRUoMHvkDtCddC 8bfxpLcOi8WMlySAud+unKnpZVwlwn2rZ/enu2Dd01QKOgOQkBr1HWTJUguPv4No eWT3UiEZhV4hU764gtF7a8raHjbvxpTJcNk22KHnRmyTKX+SugCyI0qkmIQrFntl cQWXyA9GV7V+5bK5/Sp2eWv2MXX3fhNDxtZywkmqun+6/YhPgSDJQp3FcKThZFYy WxPXsrXVIXFJbbtkSs+PGe18VdXqEFCHOI29H1+9gyiC3FW3N6AARw== =GA3y -----END PGP SIGNATURE-----
