[Distutils] Releasing eggs as .tar.gz with setuptools or zc.buildout
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, I am now working on a project that I need to deploy on different servers in a compiled form, as tar.gz files (the servers do not have gcc for example). The project comprises two libraries and an extra set of config files and scripts for batch computation. I would like to have a utility that does all the packaging for me, that is create a tar.gz for each lib and one for the scripts, each containing a directory with the compiled things. Here are my different approaches: - - setuptools bdist: This is almost what I need, except that it creates a usr dir on top of the tarball. - - zc.buildout: this looked promissing (although pretty hard to get into). I stumbled upon collective.releaser, which I thought was what I needed, but its project_deploy script just tarballs the entire buildout directory. Here is the config file I used for project_deploy: [buildout] parts = myproj [myproj] recipe=zerokspot.recipe.git repository=/data/proj/myproj branch=unstable as_egg=true I am getting quite desperate... Can anyone give me pointers ? Thanks, Martin -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJLqjFQAAoJEBdvyODiyJI4Y/cH/0/3osiqHWlzoT4lDiSgnrSu oG/mQMUHE+kOVHzDaZlf6mEQiTw0Ro2JPOwSSYlmf7dMXxmcCjkvNSmaqf2iYNE6 DN2SfMguIYG7v+hU3y9D9nPCdg4uf0kpxvTOFqm4cGkhnsqnIsy585eW6NLuXiwU pDuWlkLmYcFgVZGLczEJQ+p0h9ujigrVUZfH1vq+hn1r+Fpa7X69N2EMtBxq/Ull svqHQrNrEI3Brcv/9T/JMgen3//Mgb4vUrbN81LBa6rOJYth6AK6dBbkp5wicaUn QnPrl5/im59UJ5IT5Mrbj7o4kYdvKaxM9Iw+5fX4rnp0nV8mlbKv5vastCCJtp0= =qqAr -END PGP SIGNATURE- <>___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Releasing eggs as .tar.gz with setuptools or zc.buildout
On Wed, Mar 24, 2010 at 9:05 PM, Martin Raspaud wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Hello, > > I am now working on a project that I need to deploy on different servers in a > compiled form, as tar.gz files (the servers do not have gcc for example). > > The project comprises two libraries and an extra set of config files and > scripts > for batch computation. > > I would like to have a utility that does all the packaging for me, that is > create a tar.gz for each lib and one for the scripts, each containing a > directory with the compiled things. > > Here are my different approaches: > - - setuptools bdist: This is almost what I need, except that it creates a > usr dir > on top of the tarball. > - - zc.buildout: this looked promissing (although pretty hard to get into). I > stumbled upon collective.releaser, which I thought was what I needed, but its > project_deploy script just tarballs the entire buildout directory. > > Here is the config file I used for project_deploy: > [buildout] > parts = > myproj > > [myproj] > recipe=zerokspot.recipe.git > repository=/data/proj/myproj > branch=unstable > as_egg=true > > I am getting quite desperate... > > Can anyone give me pointers ? May be this will be useful: http://pypi.python.org/pypi/zc.sourcerelease Regards, Baiju M ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Releasing eggs as .tar.gz with setuptools or zc.buildout
On Wed, Mar 24, 2010 at 12:22 PM, Baiju M wrote: > May be this will be useful: > http://pypi.python.org/pypi/zc.sourcerelease It sounds to me like Martin's problem isn't getting source releases, but in deploying them on systems without compilers and other tools. What we're doing at Zope Corporation is building source releases (using zc.sourcerelease as Baiju suggested), and then building packages that the system package management tools can use from those source releases. (In our case the packages are RPMs, but it could be anything.) But the zc.sourcerelease releases are a good starting point for that since the dependency packaging and management is handled for you. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Releasing eggs as .tar.gz with setuptools or zc.buildout
On Wed, Mar 24, 2010 at 10:10 PM, Fred Drake wrote: > On Wed, Mar 24, 2010 at 12:22 PM, Baiju M wrote: >> May be this will be useful: >> http://pypi.python.org/pypi/zc.sourcerelease > > It sounds to me like Martin's problem isn't getting source releases, > but in deploying them on systems without compilers and other tools. > > What we're doing at Zope Corporation is building source releases > (using zc.sourcerelease as Baiju suggested), and then building > packages that the system package management tools can use from those > source releases. (In our case the packages are RPMs, but it could be > anything.) Can you give some pointer how you are creating RPMs for the binary eggs (eg:- zope.interface ) Any tool available ? These RPMs are for entire buildout or individual eggs ? Regards, Baiju M ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Releasing eggs as .tar.gz with setuptools or zc.buildout
On Wed, Mar 24, 2010 at 12:48 PM, Baiju M wrote: > Can you give some pointer how you are creating RPMs for > the binary eggs (eg:- zope.interface ) Any tool available ? The RPM is built using the normal RPM tools: we write a spec file, then run rpmbuild in what we think are the normal ways. Nothing special is needed. > These RPMs are for entire buildout or individual eggs ? For the entire buildout, since they're applications. We use a separate package (RPM) with a "clean" Python (nothing in site-packages!) that won't be affected by general system updates. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Releasing eggs as .tar.gz with setuptools or zc.buildout
On Wed, Mar 24, 2010 at 6:24 PM, Fred Drake wrote: > On Wed, Mar 24, 2010 at 12:48 PM, Baiju M wrote: >> Can you give some pointer how you are creating RPMs for >> the binary eggs (eg:- zope.interface ) Any tool available ? > > The RPM is built using the normal RPM tools: we write a spec file, > then run rpmbuild in what we think are the normal ways. Nothing > special is needed. > >> These RPMs are for entire buildout or individual eggs ? > > For the entire buildout, since they're applications. We use a > separate package (RPM) with a "clean" Python (nothing in > site-packages!) that won't be affected by general system updates. We use to do the same but we build the isolated buildout using collective.releaser. The tool is a script that creates a binary zip containing the built buildout, that can be re-run on a similar arch without any web access. It also has features like patch creation for buildout (will create a small archive with a patch to be applied on an existing buildout for upgrades) I think the tool is still maintained by Ingeniweb but under-documented > > > -Fred > > -- > Fred L. Drake, Jr. > "Chaos is the score upon which reality is written." --Henry Miller > ___ > Distutils-SIG maillist - distutils-...@python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- Tarek Ziadé | http://ziade.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Releasing eggs as .tar.gz with setuptools or zc.buildout
At 04:35 PM 3/24/2010 +0100, Martin Raspaud wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, I am now working on a project that I need to deploy on different servers in a compiled form, as tar.gz files (the servers do not have gcc for example). The project comprises two libraries and an extra set of config files and scripts for batch computation. I would like to have a utility that does all the packaging for me, that is create a tar.gz for each lib and one for the scripts, each containing a directory with the compiled things. Here are my different approaches: - - setuptools bdist: This is almost what I need, except that it creates a usr dir on top of the tarball. You could always repack the tarball without it, or change your default installation prefix to match that of the target system(s). (There's a --relative option, too, but it seems to be broken when I try it on my systems.) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Releasing eggs as .tar.gz with setuptools or zc.buildout
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Fred Drake skrev: > On Wed, Mar 24, 2010 at 12:22 PM, Baiju M wrote: >> May be this will be useful: >> http://pypi.python.org/pypi/zc.sourcerelease > > It sounds to me like Martin's problem isn't getting source releases, > but in deploying them on systems without compilers and other tools. That is it exactly. Ideally, the zc recipe I'm looking for would install the egg or develop egg to a given directory, then tarball this directory. > What we're doing at Zope Corporation is building source releases > (using zc.sourcerelease as Baiju suggested), and then building > packages that the system package management tools can use from those > source releases. (In our case the packages are RPMs, but it could be > anything.) > But the zc.sourcerelease releases are a good starting point for that > since the dependency packaging and management is handled for you. I will check it out. Thanks ! Martin -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJLqw9DAAoJEBdvyODiyJI4zYMH/R0pJqDdCnNmqVM/x5z0Xhp0 rueSukCGLl5B8rEs+e32B978g8YNe2mBAFi/Tq+bNmg11Osji+ZYhC36Xyy3LuTK fwTo+aXUszmOAy8lSEXHpCdfnR2ydxLPCwpkMJIfYUqbxsWubZ2lhFPnZzUUxp8m 68h2eNYtE3mtKFAKne/LQASAr04edLCph+BAhPZWCGZv11wm+7m+vBKTxQIefM8P 7I7N2Ni8J3sRvLj3Sq7Khl4jKanQiRD8/d1fJzRrck0dZxbKi1bpyRFtQhOnfV0B ZK+yGD/DD4hbYvyx/5pHV7yatTcfCXpidr1ppZj8oMdmBruUUVolFDM5Qv4wIfw= =SSUg -END PGP SIGNATURE- <>___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Releasing eggs as .tar.gz with setuptools or zc.buildout
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 P.J. Eby skrev: > At 04:35 PM 3/24/2010 +0100, Martin Raspaud wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> - - setuptools bdist: This is almost what I need, except that it >> creates a usr dir >> on top of the tarball. > > You could always repack the tarball without it, yes, but it would be nice if it was automated. > or change your default > installation prefix to match that of the target system(s). - --prefix and --install-dir do not work with the bdist command. > (There's a --relative option, too, but it seems to be broken when I try > it on my systems.) > Doesn't work for me either. Is it supported in distribute ? Regards, Martin -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJLqw6WAAoJEBdvyODiyJI4dKYIAKUZxFTbWJTEHB4KJXzgk6Pb i0setaMUU8xBKs0aOFNWbyWRzXrqDu3ecpWDQ8HUzFwsbO+eo6EJDExuWTxvFvo3 CNj9tvtKKiBRENcwG1dK1NyDRFyIHaxSYaCPpAy6DHpN1BXEnm6pQMeKpXUQVRek r+XKdc+hdwWyDVo9zo+4zYLrVe5rGf8F4MTYYVTNY49xcI0Jg/y7o0dS1HzkZAs2 D8kaQ8h7/PeuoNKI8e4MCLT8NRQ8VzRfFdySBG5O2foEhlLzKD/pA0epS4p0x8BC ToP6GE4Uedl9Wa8O92z4zAqQxhzp40UXU0UvRU3jvaY8Je5uKKfjX/HGpmTIhsk= =wKCU -END PGP SIGNATURE- <>___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Releasing eggs as .tar.gz with setuptools or zc.buildout
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Tarek Ziadé skrev: > On Wed, Mar 24, 2010 at 6:24 PM, Fred Drake wrote: >> On Wed, Mar 24, 2010 at 12:48 PM, Baiju M wrote: >>> Can you give some pointer how you are creating RPMs for >>> the binary eggs (eg:- zope.interface ) Any tool available ? >> The RPM is built using the normal RPM tools: we write a spec file, >> then run rpmbuild in what we think are the normal ways. Nothing >> special is needed. >> >>> These RPMs are for entire buildout or individual eggs ? >> For the entire buildout, since they're applications. We use a >> separate package (RPM) with a "clean" Python (nothing in >> site-packages!) that won't be affected by general system updates. > > We use to do the same but we build the isolated buildout using > collective.releaser. > > The tool is a script that creates a binary zip containing the built > buildout, that can be re-run > on a similar arch without any web access. > > It also has features like patch creation for buildout (will create a > small archive with a patch > to be applied on an existing buildout for upgrades) > > I think the tool is still maintained by Ingeniweb but under-documented > Indeed, the documentation isn't very clear to me. Project_deploy seemed to do almost what I wanted, except it was packaging the buildout with the egg, which is not suitable for our installation. Is there way to tell it to package just an egg ? Thanks, Martin -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJLqxAUAAoJEBdvyODiyJI4DlYIAM4t6y3/d/NEphKqUvAkHJqc ZnqBm1OJ0jLhDw7ca0pWAxz/IS5XuvVW2TxX1sO45D6Xf+8OjWm+CY5hoOFAdI3K vQbDEpCRgSCAf+NWTAeB8JKelaoHKRNY0VKeCrZ/YH8EFZi61XHGYBKBbInDhsce i6fe+B0/P/DVTcFvyQGxr3vz3mi2S9RNQ+SsAuj8Y5U9j3TIuf2uAVl16sRSZdZa j6aDwDh+3B8U3aXQuL7IYi5jYP6NkkyX4RqjZUn8DMzjqYAevuYoMqcMnkax5hCp /A+WT9vBcnVnZJIKXEYalgp1QagvxX7NfjPcayVTwhzeJFmzHT0awqmSLbg69bM= =fx/O -END PGP SIGNATURE- <>___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Releasing eggs as .tar.gz with setuptools or zc.buildout
On 03/24/2010 05:40 PM, Fred Drake wrote: On Wed, Mar 24, 2010 at 12:22 PM, Baiju M wrote: May be this will be useful: http://pypi.python.org/pypi/zc.sourcerelease It sounds to me like Martin's problem isn't getting source releases, but in deploying them on systems without compilers and other tools. What we're doing at Zope Corporation is building source releases (using zc.sourcerelease as Baiju suggested), zc.sourcerelease works fine. I've got one thing I need to fix in it: distribute support. I've got distribute installed globally and zc.sourcerelease has 'setuptools' hardcoded. Corner case. I'll probably make a branch for it, ok? and then building packages that the system package management tools can use from those source releases. (In our case the packages are RPMs, but it could be anything.) I experimented this week to create debian packages with zc.sourcerelease. Fun! I didn't finish it yet, though. There was one question that I couldn't figure out yet: *How do you get buildout to build in a subdirectory?* I mean, you can give gnu's autoconf mechanism a base prefix argument so that everything gets installed into /where/I/build/usr/lib/something instead of /usr/lib/something. If I run buildout, it uses absolute directories, so /where/I/build/ will still be in the paths. My "solution" would be to use debian to get the big .tgz into the target dir and then to run "python install.py" in a post-install hook or so. But... I'm probably missing something. As you're making RPMs you probably have a solution ready :-) Reinout -- Reinout van Rees - rein...@vanrees.org - http://reinout.vanrees.org Programmer at http://www.nelen-schuurmans.nl "Military engineers build missiles. Civil engineers build targets" ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Releasing eggs as .tar.gz with setuptools or zc.buildout
On Thu, Mar 25, 2010 at 11:30 AM, Reinout van Rees wrote: > I'll probably make a branch for it, ok? Sounds good. > *How do you get buildout to build in a subdirectory?* I mean, you can give > gnu's autoconf mechanism a base prefix argument so that everything gets > installed into /where/I/build/usr/lib/something instead of > /usr/lib/something. There's an option to get buildout to use relative paths; you'll need to dig that out of the docs and use that. Some recipes may not support that since it's (relatively) recent. (Sometime in the past year, maybe?) > If I run buildout, it uses absolute directories, so /where/I/build/ will > still be in the paths. My "solution" would be to use debian to get the big > .tgz into the target dir and then to run "python install.py" in a > post-install hook or so. We're making RPMs, but I'll bet not all of them are relocatable. I know we're using the relative paths bit in some to make things better. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
