[interesting outline implementation snipped]The biggest time suck in developing MakeMaker, and to a lesser extent Test::More, is running the tests. Why? Because they need to be run on lots of different platforms with lots of different versions of Perl. Currently, I do this by hand. And we all know manual testing sucks. Its time consuming and you tend to avoid it. I can't run the tests on every platform at every patch so I often wind up breaking something and not realizing it for a while.
So what I need is some way to set up a network of test servers such that
I can say "test this module for me" and my testing client would ship it
to as many test servers as it can find and get the results back all in
just a few minutes.
Random comments. I have zero available tuits to help implementation so feel free to ignore and/or laugh:
- Nice idea
- If this is going to be run by paranoid people everything would have to be over https to prevent man-in-the-middle attacks on the code being transported
- I've done a vaguely related task in the past. Instead of distributing one test suite over several machines, I was distributing bits of a single test suite over several machines (so I could run the test scripts in parallel and decrease the overall runtime of the whole test suite). Would be nice if we had something flexible enough to cope with both scenarios - but that might make it too complex to implement in a day or so :-)
- I solved my problem with SSH rather than HTTP since I had the infrastructure for it in place on the machines I was playing with. Might be worth considering as an alternative to HTTP[S]
- Would you want to deal with cannot test (e.g. because the test server didn't have the necessary prerequisite modules rather than the test itself timing out or there being a communication problem) as well as pass/fail?
- Some mechanism to automatically gather/report the base platforms would seem to be a good idea. Otherwise you are going to have people forgetting to keep the central list up to date when then update their box.
Cheers,
Adrian
