On Fri, Jul 4, 2008 at 6:43 PM, Marius Gedminas <[EMAIL PROTECTED]> wrote: > On Fri, Jul 04, 2008 at 05:44:12PM -0400, Benji York wrote:
>> There's no appreciable bookkeeping for the parallelization, so I don't >> know where the CPU time is going. > > Every layer is spawned in a separate subprocess, right? That means 36 > new Python processes with the associated startup cost, plus the module > import cost, plus some test result marshalling through plain-text Unix > pipes. Two seconds of startup cost per subprocess would nicely account > for the one extra minute of user time if there are over 30 subprocesses. The number of subprocesses is the same as for the trunk, the only change is that they can be spawned in parallel. Wait! The Zope 3 test layers can all be torn down! Therefore there aren't *any* subprocesses spawned normally. Ok, that makes more sense. (time passes) OK, I did a check out of the Zope 3 trunk and was able to duplicate your results. (And wow, the trunk seems to be in bad shape -- lots of tests failing. I guess it's fallen into disrepair since being broken out into subprojects.) > My crude measurements (time ./test.py --list-tests > /dev/null) indicate > the time needed to import everything is closer to 4 seconds, but that's > importing everything -- importing just the things needed for a single > layer may reduce that to two seconds on average. A possible enhancement would be to reuse subprocesses if they are asked to run layers that can be torn down. That way for a very tear-down friendly set of tests like Zope 3's, the minimum number of processes would be started. We could also use fork to eliminate some of the start-up cost, but that's not real attractive, being un-Windows-friendly. -- Benji York Senior Software Engineer Zope Corporation _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )