On Mon, Feb 05, 2007 at 03:24:37AM +0000, Andy Armstrong wrote:
> On 5 Feb 2007, at 01:08, Eric Wilhelm wrote:
> >I was just thinking that since my tests are nicely orthogonal and yet
> >there's very little cpu utilization during testing, that I could
> >probably get them done about 20x faster if they were parallelized.
>
> [snip]
>
> I can't say off the top of my head what problems might arise.
> Obviously the output displayed during test runs would have to change.
> Sounds like a good idea though - I've added it to the TODO list :)
I had an experiment with making Test::Harness run in parallel. The code
exists, but is (intentionally) very hacky - I wanted to see what needed
doing.
>From memory, and I don't know how TAPx::Harness differs, currently when you
run a test under Test::Harness it does roughly the following:
1: Print out the test name with the ....s
2: Start the test running
3: Suck TAP data from the running test and print out progress
4: When the test completes print out its summary
5: repeat
To run tests in parallel, what I was doing was starting a farm of tests,
and when a test completed, printing out its results and starting another
one, until there were no new tests left to start, at which point wait for
tests to finish. The farm was run via a select loop.
So the logic had to change to roughly (and lets see if I can get the syntax
right):
async {
1: Start a test running
2: Suck TAP data from the test and buffer it
3: When the test completes process it all
4: Print out a summary
5: repeat
}
Important differences were
1: you only print out the entire test results as an atomic line when it has
finished
2: You have no idea which test is going to finish next
3: You can't print running progress on tests
Ideally the same code can run in both modes, so that when you aren't running
tests in parallel (which includes the case of waiting-for-the-last-test)
you go back to printing out the name.... and processing TAP as it appears.
Something I discovered was that processing TAP is slow. Up to about 7 CPUs,
parallelising tests feeding TAP into the master "thread" was a win. Beyond
that, the process processing the TAP was the bottleneck, and adding more
CPUs (and more parallel tests) didn't help. I would assume that from that
point on it would be better to have each test run by slave harness processes,
which takes the TAP, parses it, and slings (say) Storable objects across
to the master. But right now this isn't an issue as the 32 core Sun isn't
powered up. :-(
For the actual running of tests I tweaked runtests() so that the list of
arguments was treated as follows
1: Plain file names were tests to run in parallel, in any order
2: Array references were groups of tests to run in series
(This let me run tests for each module in ext/ and lib/ in order)
3: There was a special sync object that meant "complete everything up to here"
which was sufficient to give a lot of flexibility in how tests were run
(and discover that the core is very unoriginal in its naming of temporary
files)
Nicholas Clark
