On Sat, Jan 21, 2012 at 02:19:03AM -0800, Ross Patterson wrote: > I moved this patch to a branch of zc.buildout: > > svn+ssh://svn.zope.org/repos/main/zc.buildout/branches/env-cache > [...snip bugfixing...]
> I also compared buildout run times (using the 'time' command, not > cProfile) on a real world buildout with 6 identical parts and a few > other parts with very similar distribution requirements. The time > without the patches was 1m27.513s and with the patches as applied to > zc.buildout 1.4.4 it was 0m34.674s. Ouch. My slowest buildout takes "only" 41 seconds to do nothing. Still, I'd welcome a 3x improvement. > Also, buildout.dumppickedversions > suffers the same logging hot spot as I fixed in zc.buildout r122980 and > before I patched it the buildout run time was 2m13s. Can someone cut a > release of buildout.dumppickedversions? There's also buildout-versions [1], which is a fork/reimplementation of buildout.dumppickedversions, and which I use for my buildouts. I wonder if the same speedup could be applied to it. Perhaps not: if I disable buildout-versions in my 41s do-nothing buildout, the do-nothing time stays exactly the same. [1] http://pypi.python.org/pypi/buildout-versions Incidentally, looking at r122980 I wonder if you got the right revision number [2]? It seems to be a bugfix for LP#697913 (Buildout doesn't honor exit code from scripts). [2] http://zope3.pov.lt/trac/changeset/122980 > All told, this is a 4 fold real world improvement. With all those hot > spots addressed there are no obvious wastes I can find in the profiling > data. I'd like to merge this into trunk and the 1.4 branch and see > releases cut of both 1.5 and 1.4. May I begin the merging? I wish Jim Fulton would found some time to look at your branch. 4x real-world speedup seems very compelling. How can I test your branch? It doesn't seem to be enough to add [buildout] develop += /path/to/your/branch/checkout [versions] zc.buildout = to my existing buildout.cfg. Marius Gedminas -- HOST SYSTEM NOT RESPONDING, PROBABLY DOWN. DO YOU WANT TO WAIT? (Y/N)
signature.asc
Description: Digital signature
_______________________________________________ Distutils-SIG maillist - [email protected] http://mail.python.org/mailman/listinfo/distutils-sig
