Smoking private dists?
Hi!
I've recently started a new job (and a small company (which btw explains
a certain CPANTS lagging on my part...)), and we want to smoke test our
code.
Are there any tools/frameworks available that would (in a perfect, lazy
world) to the following:
* fetch various projects from git (HEAD, and some defined tags)
* run their test suite
* run Devel::Cover (eg via Build testcover)
* run some special tests (eg Pod::Coverage), which probably means to run
some tests in the testsuite with some special ENV settings
* report all results on a nice and colorfull website
If there isn't, are there any frameworks to customize, or CPAN modules
to glue together?
Smolder looks interesting, but it seems to be more concerned with
presenting test results and so much with the acutal smoking (or am I
missing something)?
Then there's SmokeRunner::Multi, but I couldn't understand how it's
supposed to work from the docs (and before I burn a few hours playing
with it, I'd rather ask here...)
And I presume that the various CPAN smoking tools don't fit my needs,
because they are designed to either smoke the whole of CPAN, or to send
back test results after installing a given CPAN dist.
Therefore, my current approach would be to hack together a few scripts
that fetch stuff from git, run the tests, collect the results, and
report them.
Unless somebody shows me the one wellknown package that
does what I need and that everybody besides me knows about :-)
Thanks!
--
#!/usr/bin/perl http://domm.plix.at
for(ref bless{},just'another'perl'hacker){s-:+-$-gprint$_.$/}
Re: Smoking private dists?
Thomas Klausner wrote: And I presume that the various CPAN smoking tools don't fit my needs, because they are designed to either smoke the whole of CPAN, or to send back test results after installing a given CPAN dist. Remember that you can always bind your own local view of the CPAN with CPAN::Mini and CPAN::Mini::Inject » http://search.cpan.org/dist/CPAN-Mini-Inject/ -- Sébastien Aperghis-Tramoni Close the world, txEn eht nepO.
Re: Smoking private dists?
On 5 Feb 2008, at 08:58, Thomas Klausner wrote: Are there any tools/frameworks available that would (in a perfect, lazy world) to the following: * fetch various projects from git (HEAD, and some defined tags) * run their test suite * run Devel::Cover (eg via Build testcover) * run some special tests (eg Pod::Coverage), which probably means to run some tests in the testsuite with some special ENV settings * report all results on a nice and colorfull website You could look at buildbot. It can do all of the above. It's in Python if that matters :) -- Andy Armstrong, Hexten
Re: Smoking private dists?
On Feb 5, 2008 11:55 AM, Andy Armstrong [EMAIL PROTECTED] wrote: You could look at buildbot. It can do all of the above. It's in Python if that matters :) Or Tinderbox which was the original that buildbot copied. (Why did they have to keep the GUI I don't know :) Gabor
Re: Smoking private dists?
On Feb 5, 2008, at 2:31 AM, Gabor Szabo [EMAIL PROTECTED] wrote: On Feb 5, 2008 11:55 AM, Andy Armstrong [EMAIL PROTECTED] wrote: You could look at buildbot. It can do all of the above. It's in Python if that matters :) Or Tinderbox which was the original that buildbot copied. And now buildbot will be replacing tinderbox.
Re: Smoking private dists?
On 5 Feb 2008, at 14:44, Philippe Bruhat (BooK) wrote: That's what I did at $work. I store all the results in a database, and hope to come up with creative reports (especially historical ones) in the future. TAP::Parser was very helpful in collecting the test results. Cool! Nice to hear of someone using it :) -- Andy Armstrong, Hexten
Re: Smoking private dists?
On Tue, Feb 05, 2008 at 09:58:39AM +0100, Thomas Klausner wrote: Therefore, my current approach would be to hack together a few scripts that fetch stuff from git, run the tests, collect the results, and report them. That's what I did at $work. I store all the results in a database, and hope to come up with creative reports (especially historical ones) in the future. TAP::Parser was very helpful in collecting the test results. My smoke script is basically fetching the list of all test files, then looping over each of them with TAP::Parser, and producing various objects (stored in a db) for smoke / script / test. The biggest trouble I had was for diagnostics. I ended up considering that diagnostics output after a test result belong to the test result (as a comment to it), and that diagnostics appearing before the first test result are global to the whole test script. Which means that # Looks like you failed 3 tests of 22. is always attached to the last test. -- Philippe Bruhat (BooK) You are never too old to have your seat tanned by your Grandmother. (Moral from Groo The Wanderer #41 (Epic))
Testing on live systems
A cautionary tale of why we must be very careful doing tests against live systems. http://thedailywtf.com/Articles/Ive-Got-The-Monkey-Now.aspx -- Andy Lester = [EMAIL PROTECTED] = www.petdance.com = AIM:petdance
Re: Update 3 on the QA Hackathon in Oslo
Hi Salve and others. I have added an entry to http://adserver.szabgab.com/ advertising the QA workshop. It is in English and should show up in every country. If you can send me texts in other languages as well with the appropriate country code I'd be glad to add them. See the linked yaml file to get ideas http://adserver.szabgab.com/ads.yml regards Gabor
