Re: Double build failures
On Thu, May 10, 2012 at 12:07:28PM +0200, Jakub Wilk wrote: > * Brian Sutherland , 2012-05-10, 10:36: > >AFAIK, the problem is double build failures because dpkg detects > >that there are leftovers/changes from the first build. You end up > >with failures when a second build is attempted [1]. > > > >This leads people to use workarounds like this snippet taken from > >the style guide [2]: > > > > override_dh_auto_clean: > > dh_auto_clean > > rm -rf build > > rm -rf *.egg-info > > (Note that "rm -rf build" is unrelated to egg-info problem.) > > There are simpler workarounds, though: > > 1) Add *.egg-info/* to debian/clean. (This requires compat >= 7.) > > 2) Add extend-diff-ignore="^[^/]+\.egg-info/" to debian/source/options. > (This requires source format 3.0.) > > >Which work sometimes, but sometimes cause other bugs with the > >second build. For example, when setup.py uses > >setuptools+include_package_data without specifying a MANIFEST.in. > >In that case deleting *.egg-info causes missing files in the built > >package. > > Interesting. Can you show us example of a package like this? For example zope.configuration, or zope.security, had this problem around Nov 2011. Even running the tests didn't help as they ran at build time against the unpacked source. > In any case, solution 2) wouldn't trigger this problem. Yeah, thats the approach that was used in the end for the zope.* packages. Any objections (from anyone) to updating the style guide to prefer solution 2? -- Brian Sutherland -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120510143708.gb24...@minimac.lan
Re: Double build failures
* Brian Sutherland , 2012-05-10, 10:36: AFAIK, the problem is double build failures because dpkg detects that there are leftovers/changes from the first build. You end up with failures when a second build is attempted [1]. This leads people to use workarounds like this snippet taken from the style guide [2]: override_dh_auto_clean: dh_auto_clean rm -rf build rm -rf *.egg-info (Note that "rm -rf build" is unrelated to egg-info problem.) There are simpler workarounds, though: 1) Add *.egg-info/* to debian/clean. (This requires compat >= 7.) 2) Add extend-diff-ignore="^[^/]+\.egg-info/" to debian/source/options. (This requires source format 3.0.) Which work sometimes, but sometimes cause other bugs with the second build. For example, when setup.py uses setuptools+include_package_data without specifying a MANIFEST.in. In that case deleting *.egg-info causes missing files in the built package. Interesting. Can you show us example of a package like this? In any case, solution 2) wouldn't trigger this problem. -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120510100728.ga5...@jwilk.net
Re: Double build failures
On Fri, May 04, 2012 at 01:28:57PM -0400, Éric Araujo wrote: > Hi list, > > Le 04/05/2012 13:23, Sandro Tosi a écrit : > >CPython upstreams are developing a new module to replace distutils and > >setuptools: packaging. It might be worth check with them if it will > >handle this case. > > Can you explain what the problem is? (I never packaged something > that uses setuptools.) > > In distutils2/packaging, a dist-info directory is created (successor > of the egg-info dir, see PEP 376) on installation, and it includes a > file named RECORD with paths to installed files. The clean command > only removes build by-products though, so it wouldn’t remove the > dist-info dir by design (just as now it does not remove the egg-info > file or dir). Hmm, no one responded to Éric, so I'll give it a try. Feel free to correct me if I don't explain enough or get it wrong (as is likely). AFAIK, the problem is double build failures because dpkg detects that there are leftovers/changes from the first build. You end up with failures when a second build is attempted [1]. This leads people to use workarounds like this snippet taken from the style guide [2]: override_dh_auto_clean: dh_auto_clean rm -rf build rm -rf *.egg-info Which work sometimes, but sometimes cause other bugs with the second build. For example, when setup.py uses setuptools+include_package_data without specifying a MANIFEST.in. In that case deleting *.egg-info causes missing files in the built package. I think that most people here are arguing that "setup.py clean" should cleanup all build products and get you back to some kind of pristine state. [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=645186 [2] http://wiki.debian.org/Python/LibraryStyleGuide -- Brian Sutherland -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120510083624.ga12...@minimac.lan
Re: Double build failures
On Fri, 04 May 2012, Jakub Wilk wrote:
> * Yaroslav Halchenko , 2012-05-04, 13:29:
> >>>Most of the time, the failures are because of the created egg
> >>>directory that is not cleaned up by the clean target of
> >>>setup.py. While I could add the appropriate bits to each clean
> >>>target in debian/rules, it seems better to fix the problem
> >>>either in dh_python2 or in setuptools. Any thoughts?
> >>IMHO it is a better fit in dh_auto_clean (python_distutils
> >>buildsystem) and or dh_clean.
> >+100 for dh_auto_clean ;-)
> #652617
> But... Have you read python_distutils.pm source? I did and then
> couldn't sleep for a few nights. ;) So I can't second the idea of
> adding even more dubious logic to it.
LOL
I just don't know perl (un)fortunately to provide a clean patch... nor out of
my ignorance I am sure on the purpose of distributing .egg-info/ by
upstreams
$> zgrep .egg-info deb/docs/Contents-sources.gz | f1 | uniq | wc -l
365
so the question would also be either they get modified at package build-time so
that original versions would need to be placed aside (similar to what
dh_autoreconf does?) and then reverted back upon clean. So general resolution
could go a bit aside of just 'cleaning' through
* preserve present .egg-info somewhere (where below find would not find ;) )
* add to sub clean
$this->doit_in_sourcedir('find', '.', '-name', '*.egg-info', '-exec', 'rm',
'-rf', '{}', ';');
* bring them back and destroy the original copy
or if nothing get modified just remembering the list of them and
excluding them from clean up procedure
--
Yaroslav O. Halchenko
Postdoctoral Fellow, Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik
--
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20120504183023.gq9...@onerussian.com
Re: Double build failures
On May 04, 2012, at 07:16 PM, Thomas Kluyver wrote: >On 4 May 2012 19:06, Dmitrijs Ledkovs wrote: >> ok. what is the relationship between 'distribute' & 'packaging'? > >Let's see if I get all these right: > >distutils: basic packaging functionality, part of the Python standard library >setuptools: third party module to add functionality that distutils lacked >distribute: continuation of setuptools after setuptools development stopped >packaging, aka distutils2: Improving on distutils but not backwards >compatible, will be integrated into the standard library for Python >3.3, and might then make setuptools/distribute obsolete. > >Please correct me if I've got any of that wrong. Well done! A few minor corrections. - packaging is available in Python 3.3 (alpha) now. - distutils2 is the backport of packaging to older Pythons, available in the Cheeseshop. Actually, the Cheeseshop entry has a pretty good explanation: http://pypi.python.org/pypi/Distutils2 Cheers, -Barry -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120504142850.233c9...@resist.wooz.org
Re: Double build failures
On 4 May 2012 19:06, Dmitrijs Ledkovs wrote: > ok. what is the relationship between 'distribute' & 'packaging'? Let's see if I get all these right: distutils: basic packaging functionality, part of the Python standard library setuptools: third party module to add functionality that distutils lacked distribute: continuation of setuptools after setuptools development stopped packaging, aka distutils2: Improving on distutils but not backwards compatible, will be integrated into the standard library for Python 3.3, and might then make setuptools/distribute obsolete. Please correct me if I've got any of that wrong. Thomas -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAOvn4qiG6-kqWUh2vsHV03=6hoY9=sqy8ryo-rh2+xrvo3e...@mail.gmail.com
Re: Double build failures
On 04/05/12 19:04, Scott Kitterman wrote: > On Friday, May 04, 2012 06:38:30 PM Dmitrijs Ledkovs wrote: >> On 04/05/12 18:23, Sandro Tosi wrote: >>> On Fri, May 4, 2012 at 7:19 PM, Jakub Wilk wrote: > or in setuptools. Ideally, by burning it with fire. >>> >>> CPython upstreams are developing a new module to replace distutils and >>> setuptools: packaging. It might be worth check with them if it will >>> handle this case. >> >> I swear last week it was called distribute... >> >> (if a buildsystem changes it's name more often than every 10 years think >> twice before using it.) > > distribute is a fork of setuptools. distutils is still distutils, so you can > use it only thinking once. > ok. what is the relationship between 'distribute' & 'packaging'? -- Regards, Dmitrijs. signature.asc Description: OpenPGP digital signature
Re: Double build failures
On Friday, May 04, 2012 06:38:30 PM Dmitrijs Ledkovs wrote: > On 04/05/12 18:23, Sandro Tosi wrote: > > On Fri, May 4, 2012 at 7:19 PM, Jakub Wilk wrote: > >>> or in setuptools. > >> > >> Ideally, by burning it with fire. > > > > CPython upstreams are developing a new module to replace distutils and > > setuptools: packaging. It might be worth check with them if it will > > handle this case. > > I swear last week it was called distribute... > > (if a buildsystem changes it's name more often than every 10 years think > twice before using it.) distribute is a fork of setuptools. distutils is still distutils, so you can use it only thinking once. Scott K -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4740075.3CZbIJrSJo@scott-latitude-e6320
Re: Double build failures
* Yaroslav Halchenko , 2012-05-04, 13:29: Most of the time, the failures are because of the created egg directory that is not cleaned up by the clean target of setup.py. While I could add the appropriate bits to each clean target in debian/rules, it seems better to fix the problem either in dh_python2 or in setuptools. Any thoughts? IMHO it is a better fit in dh_auto_clean (python_distutils buildsystem) and or dh_clean. +100 for dh_auto_clean ;-) #652617 But... Have you read python_distutils.pm source? I did and then couldn't sleep for a few nights. ;) So I can't second the idea of adding even more dubious logic to it. -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120504180328.ga6...@jwilk.net
Re: Double build failures
On 04/05/12 18:23, Sandro Tosi wrote: > On Fri, May 4, 2012 at 7:19 PM, Jakub Wilk wrote: >>> or in setuptools. >> >> Ideally, by burning it with fire. > > CPython upstreams are developing a new module to replace distutils and > setuptools: packaging. It might be worth check with them if it will > handle this case. > I swear last week it was called distribute... (if a buildsystem changes it's name more often than every 10 years think twice before using it.) -- Regards, Dmitrijs. signature.asc Description: OpenPGP digital signature
Re: Double build failures
Hi list, Le 04/05/2012 13:23, Sandro Tosi a écrit : CPython upstreams are developing a new module to replace distutils and setuptools: packaging. It might be worth check with them if it will handle this case. Can you explain what the problem is? (I never packaged something that uses setuptools.) In distutils2/packaging, a dist-info directory is created (successor of the egg-info dir, see PEP 376) on installation, and it includes a file named RECORD with paths to installed files. The clean command only removes build by-products though, so it wouldn’t remove the dist-info dir by design (just as now it does not remove the egg-info file or dir). Regards -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fa411d9.9020...@netwok.org
Re: Double build failures
On Fri, 04 May 2012, Dmitrijs Ledkovs wrote: > > Most of the time, the failures are because of the created egg directory > > that is not cleaned up by the clean target of setup.py. While I could > > add the appropriate bits to each clean target in debian/rules, it seems > > better to fix the problem either in dh_python2 or in setuptools. Any > > thoughts? > IMHO it is a better fit in dh_auto_clean (python_distutils buildsystem) > and or dh_clean. +100 for dh_auto_clean ;-) -- Yaroslav O. Halchenko Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120504172943.ge9...@onerussian.com
Re: Double build failures
On Fri, May 4, 2012 at 7:19 PM, Jakub Wilk wrote: >> or in setuptools. > > Ideally, by burning it with fire. CPython upstreams are developing a new module to replace distutils and setuptools: packaging. It might be worth check with them if it will handle this case. Regards, -- Sandro Tosi (aka morph, morpheus, matrixhasu) My website: http://matrixhasu.altervista.org/ Me at Debian: http://wiki.debian.org/SandroTosi -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cab4xwxxtetmgztyhxikoc6zqxr+mmaw87-mfof_rah-5oy6...@mail.gmail.com
Re: Double build failures
* Vincent Bernat , 2012-05-04, 19:05: I got some bug reports from Jakub about double build failures. I have always found those kind of tests a bit silly, but that's just my opinion. Most of the time, the failures are because of the created egg directory that is not cleaned up by the clean target of setup.py. While I could add the appropriate bits to each clean target in debian/rules, it seems better to fix the problem either in dh_python2 It's none of dh_python2's business. or in setuptools. Ideally, by burning it with fire. -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120504171955.ga4...@jwilk.net
Re: Double build failures
On 04/05/12 18:05, Vincent Bernat wrote: > Hi! > > I got some bug reports from Jakub about double build failures. I have > always found those kind of tests a bit silly, but that's just my > opinion. > > Most of the time, the failures are because of the created egg directory > that is not cleaned up by the clean target of setup.py. While I could > add the appropriate bits to each clean target in debian/rules, it seems > better to fix the problem either in dh_python2 or in setuptools. Any > thoughts? IMHO it is a better fit in dh_auto_clean (python_distutils buildsystem) and or dh_clean. -- Regards, Dmitrijs. signature.asc Description: OpenPGP digital signature
Double build failures
Hi! I got some bug reports from Jakub about double build failures. I have always found those kind of tests a bit silly, but that's just my opinion. Most of the time, the failures are because of the created egg directory that is not cleaned up by the clean target of setup.py. While I could add the appropriate bits to each clean target in debian/rules, it seems better to fix the problem either in dh_python2 or in setuptools. Any thoughts? -- Vincent Bernat ☯ http://vincent.bernat.im Don't just echo the code with comments - make every comment count. - The Elements of Programming Style (Kernighan & Plauger) pgpezY72eVhuG.pgp Description: PGP signature
