2010/10/6 Łukasz Nowak <[email protected]>: > Hello, > > I am playing with zc.buildout for some time. I experienced issues with > system provided pythons (even virutalenved), so I switched to build my > own python. Of course as I am lazy, I am using buildout to configure > and compile various python versions on my machine. > > Quite often I need to provide "self contained" buildout profile, which > would be able to run using python compiled by itself. > > So my example profile looks like this: > > [buildout] > parts = > rebootstrap2.4 > realrun > > find-links = http://www.nexedi.org/static/packages/source/ > > [python2.4] > prefix = ${buildout:parts-directory}/${:_buildout_section_name_} > executable = ${:prefix}/bin/python2.4 > > # this is just wrapper for hexagonit.recipe.cmmi which makes it not > returning python part > recipe = erp5.recipe.cmmisafe > url = > http://python.org/ftp/python/2.4.6/Python-2.4.6.tgz > configure-options = > --enable-unicode=ucs4 > --with-threads > --with-readline > --with-dbm > --with-zlib > --with-ssl > --with-bz2 > > [rebootstrap2.4] > recipe = zc.recipe.egg > eggs = zc.buildout > python = python2.4 > > [realrun] > recipe = plone.recipe.command > command = > echo Running with python ${buildout:executable} > update-command = ${:command} > > So first step for me is to bootstrap builodut: > > curl -s > http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py > | python -S - > > So now my bin/buildout is referring to "system python": > head -n1 bin/buildout > #!/home/luke/bin/python -S > > Then I need to rebootstrap it: > > bin/buildout install python2.4 rebootstrap2.4 > > python2.4* parts are installing and compiling python, rebootstrap2.4 > part is reinstalling bin/buildout using this python. > > Cool, my bin/buildout is using python it compiled: > > head -n1 bin/buildout > #!/home/luke/pybuildout/parts/python2.4/bin/python2.4 > > So I can run again, this time usually. python will be compiled, as > buildout signature changed, but this is not an issue (ccache + distcc > makes it really fast). Buildout is using python he provided, > chicken-and-egg issue solved by hand: > > bin/buildout > ... > > realrun: Running ' > echo Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4' > Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4 > Unused options for realrun: 'update-command'. > > Re running builodut does not affect python anymore: > > bin/buildout > Updating python2.4-dbm-patch. > Updating python2.4. > Updating rebootstrap2.4. > Updating realrun. > realrun: Running > echo Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4 > Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4
Sounds cool! > And so far I am happy, I am using correct python version. > > What I'd like to develop is to make those steps automatically, so that > after running: > > curl -s > http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py > | python -S - > bin/buildout > > Such steps would happen automagically: > > 1 normal bootstrap with system python > 2 compile python as needed > 3 install new bin/buildout > 4 re-run bin/buildout > 5 compile python as signature change > 6 install new bin/buildout (which is the same) > 7 re-run bin/buildout > 8 do not compile python, as signature had not changed, continue... > > Theoretically steps 5,6 and 7 could be avoided, but lets say I am > purist -- I do not trust that python compiled by buildout which is > running python I do not trust is bad :) > > Easy question: Is there ability for buildout to do smart rebootstrap > in one run, with selecting part which contains python? No. It might be worthwhile to build in support for this use case. > If no I am ready to develop such thing. Cool. > So I read a bit about > extensions - I can hook before and after buildout is run. Extensions > have access to buildout object, I think they can play with buildout > externals a lot, including re-running buildout (I saw in output that > buildout re-runs itself after upgrading himself). > > Are extension the way to go with such thing? Extensions are a valuable experiment, but they aren't really fleshed out. The mechanism is very simple, which is good. To make it more valuable would require providing more hook points in buildout itself. As you point out though, you could convievably use the unload extension to cause the buildout to be rerun. > If needed I am even ready to play a bit more with zc.buildout > internals code to extend if required, but according to my > understanding how buildout works there are enough places I can "hook" > to. You may be right. It sounds like a good next step. You could also propose additional apis that might help your extension. > Any other comments? Only that your use case is not that uncommon. Perhaps there should be a built in option to have buildout build it's own Python. One thought is that building Python is expensive enough that you'd like to share the work accross buildouts. *Maybe* it would be interesting to point buildout at a separate Python buildout. It would look at the Python buildout and build *it* if necessary and then use the resulting Python. Jim -- Jim Fulton _______________________________________________ Distutils-SIG maillist - [email protected] http://mail.python.org/mailman/listinfo/distutils-sig
