Jed Brown emailed the following on 17/09/12 23:49: > On Mon, Sep 17, 2012 at 5:28 PM, Gerard Gorman > <g.gorman at imperial.ac.uk <mailto:g.gorman at imperial.ac.uk>> wrote: > > So if I understand you correctly I should start a python session > (handy > since buildbot is all python), import in your test harness and then > start calling tests. If this is the case (kick it back if I've > misunderstood) then iterating through a list of tests would seem to be > the way to go. This would give maximum flexibility to filter tests > into > categories (for example MPI tests might need to be handled differently > than serial tests). > > > So I'm planning to provide a rich API for filtering/selection. I see > that as a key motivation for doing this system in the first place. Of > course you can run a query and get back a list of test objects, but > you could also just say "run everything matching this query and report > the result". In that case, you would get called back with each > completed test. > > > This also would give plenty of flexibility to do > whatever with the output - eg might write out all > -log_summary/stdout/stderr to buildbot log in case of error and drop > otherwise. > > > The callback will be provided with all that information. > > > Have you a plan for how to handle machine specific ways to run > mpiexec/queueing etc? Are you going to recycle methods from the > BuildSystem or will there be a mechanism to supply my own mpiexec or > batch queueing harness? > > > There will be a runner interface. You'll be able to write a plugin to > run tests using some custom submission system. It will report back > return code, stderr, and stdout. >
Sounds great. > In terms of Python library dependencies, I would really like to use > gevent. It's portable and provides a very good parallel abstraction. > It does use a (tiny) C extension module. Is that dependency going to > be a deal-breaker? Distributed with Ubuntu/debian so cannot see any problem with this. > Since the testing dependency graph can (and generally will) be > constructed without needing to do anything asynchronously, the gevent > issues above are not nearly as acute. We could use a basic thread pool > and have each thread take items out of the queue. The GIL is released > when they do IO so this is fine. For configure, the dependency graph > is potentially dynamic, in which case a system that can manage many > suspended contexts would be ideal. I've done simple implementations > with several systems and the gevent code is both cleanest and fastest > so that's what I've started with. Sounds good. Let me know when there is a branch I can try hooking up. Cheers Gerard
