Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
2009/10/7 Tres Seaver tsea...@palladion.com: The fact that the bugfix (to distutils) was committed by folks who are *alos* pusing a fork to setuptools is what raises the eyebrows here. Eh... why? Tarek has become the lead for Python packaging programs and is trying hard to fix the sad state of Python packaging. It's not particularily surprising that he (or otehrs involved in Python packaging) would be involved in more than one package. Note that I am *not* suggesting that anyone acted in bad faith; but the appearance of impropriety can be present where no actual impropriety exists. The only cure is for the conflicted parties to take extreme measures to avoid appearing to act improperly. ... There are those postig on this thread that the distutils breakage is a good thing for Distribute, because it *forces* folks to switch: I call that a solid hint, myself. I'm sorry, but I read this as you implying that somebody could think Tarek, me and other Distribute people is involved in some sort of Python packaging conspiracy. I can't take that seriously, so I really hope that you mean something else, because that idea is just laughable. There is no appearance of impropriety. There is no conflicted party. There is no conflict. You can't read anything as being good for Distribute, because there is no good for Distribute. Distribute does not benefit from people using it. People benefit from using Distribute. I suspect that you see some conflict or competition between Distribute and setuptools when there is none. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On 07.10.2009, at 01:08, Sridhar Ratnakumar wrote: On Tue, 06 Oct 2009 10:25:04 -0700, kiorky kio...@cryptelium.net wrote: The fork was started by Philip Jenvey at http://bitbucket.org/pjenvey/virtualenv-distribute/ and this version by Florian Schulze lives at http://bitbucket.org/fschulze/virtualenv-distribute/ [1] - http://pypi.python.org/pypi/virtualenv-distribute/ Just a suggestion: could someone make that /fschulze/virtualenv- distribute site (the current 'home page' of the project) link to the launchpad bug tracker? Up until now, I was not able to track down an issue tracker for this fork .. until you posted the PyPI link. This way one - who stumbled upon the bitbucket site - does not have to pull the source tree and look in docs/index.rst in order to get the URL to the bug tracker. (I had a bug to report a couple of days). Ah, found the setting at bitbucket to enable the issue tracker, thanks kiorky to make me look deeper. http://bitbucket.org/fschulze/virtualenv-distribute/issues/?status=newstatus=open Regards, Florian Schulze ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Oct 7, 2009, at 3:18 AM, Florian Schulze wrote: This way one - who stumbled upon the bitbucket site - does not have to pull the source tree and look in docs/index.rst in order to get the URL to the bug tracker. (I had a bug to report a couple of days). The launchpad bugtracker is for virtualenv, not this fork. Sadly I just realized bitbucket.org doesn't have an integrated issue tracker like github :( Uh...it's cleverly hidden under the Issues tab on the project page. S ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On 07.10.2009, at 16:21, sstein...@gmail.com wrote: On Oct 7, 2009, at 3:18 AM, Florian Schulze wrote: This way one - who stumbled upon the bitbucket site - does not have to pull the source tree and look in docs/index.rst in order to get the URL to the bug tracker. (I had a bug to report a couple of days). The launchpad bugtracker is for virtualenv, not this fork. Sadly I just realized bitbucket.org doesn't have an integrated issue tracker like github :( Uh...it's cleverly hidden under the Issues tab on the project page. I had to enable it in the cleverly hidden Admin tab first. Regards, Florian Schulze ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Oct 7, 2009, at 10:51 AM, Florian Schulze wrote: On 07.10.2009, at 16:21, sstein...@gmail.com wrote: On Oct 7, 2009, at 3:18 AM, Florian Schulze wrote: This way one - who stumbled upon the bitbucket site - does not have to pull the source tree and look in docs/index.rst in order to get the URL to the bug tracker. (I had a bug to report a couple of days). The launchpad bugtracker is for virtualenv, not this fork. Sadly I just realized bitbucket.org doesn't have an integrated issue tracker like github :( Uh...it's cleverly hidden under the Issues tab on the project page. I had to enable it in the cleverly hidden Admin tab first. Told'ya it was cleverly hidden. ;-) I forgot about that; I think I had exactly the same Where the heck's the issue tracker? moment when I started my first repository. Why it's not enabled by default is one of the deep mysteries... Glad you found it! S ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On 5 Oct, 2009, at 13:54, Tarek Ziadé wrote: On Mon, Oct 5, 2009 at 1:30 PM, Ronald Oussoren ronaldousso...@mac.com wrote: Nobody will adopt it until they are forced to. This unfortunate bug means people are forced to quicker than expected. I don't think that's an actual problem. This is a problem, it means 2.6.3 is not a simple drop-in replacement for 2.6.2 but requires the replacement of another component as well. That can be a problem in organizations with strict configuration management where you cannot install new software without going to lots of red tape (and that might involve lawyers when you install a new package instead of upgrading an existing one). What is your solution ? Setuptools is not part of the standard library and monkey-patch Distutils. Its development has been discontinued for over a year. I don't have a real solution, beyond documentation. IMHO the issue should have been documented in the 2.6.3 release notes, which assumes that the issue was know before the release. Everytime Distutils is changed for anything, wether it's a bug fix or not, Setuptools can be broken. Now I realize that some folks wants Distutils to be aware of that and be backward-compatible with Setuptools monkey-patches ? I'd prefer if distutils didn't break setuptools in the 2.6 branch, breaking it in 2.7 is fine because setuptools hooks into distutils at a very low level and is therefore sensitive too changes in implementation details of distutils. And if breaking setuptools in the 2.6 branch is unavoidable this should be noted in Python's release notes. Whether we like it or not setuptools, and easy_install in particular, is used a lot and not only by power users. Anyway, the issue is less relevant at the moment the NEWS file in the 2.6 branch says that a fix for the setuptools breakage will be in 2.6.4 when that's released. Like for the svn 1.6 compat problem we had earlier this year, this problem is a few line changes in Setuptools, it's an 1 hour work. If your company upgrades to Python 2.6.3 it can also upgrade to an hypothetical Setuptools 0.6c..10 ? Sure, but 0.6c10 is only hypothetical at the moment. BTW. This doesn't mean that I don't appreciate distribute, I'm switching to that in the near future because there is real progress in distribute and it has support for py3k. Ronald ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Tarek Ziadé wrote: The proper answer is : Setuptools is on the top of Distutils and has to evolve with it. And since it monkey patches it, it has to be changed when a Distutils release breaks it. I want to note that the issue here is not monkey-patching, it is subclassing the command classes. If two tools like setuptools each subclass a distutils command, they cannot work together, it is impossible by the very 'design' of distutils, that's not setuptools' fault. numpy.distutils has the same problem (and I am sure tools like paver or pip as well): everytime distutils changes its implementation, some things are broken and we have to adapt. The only way to make this work is to test everything combination and making sure each extension does not walk on each other - that's one of the core reason why distutils is such a mess IMO. cheers, David ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
P.J. Eby kirjoitti: At 11:53 AM 10/5/2009 +0200, Lennart Regebro wrote: 2009/10/5 Jeff Rush j...@taupro.com: Very unfortunate, as in, it should NOT have happened. And *especially* without any announcement on python.org or mention on the python-committers list of something this major. Well this major... It's a bug fix that breaks a setuptools monkey-patch. Subclassing distutils commands != monkeypatching. If, say, numpy's distutils extensions subclass build_ext and override that method, they could have had the same problem. Same for anybody else's distutils extensions. Setuptools subclasses distutils commands and then replaces the original classes with its own. Example from setuptools/__init__.py: import distutils.core distutils.core.Command = Command Isn't this exactly what monkey patching means? ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
K. Richard Pixley wrote: Alex Grönholm wrote: There is a lack of consensus regarding how exactly they should work. If we are having this much trouble deciding how a third party tool should work, it is certainly not going to be merged into distutils until those issues have been resolved. Distutils is what houses (or should) the parts we all agree on. That said, I think that plenty of setuptools/distribute functionality should be moved to distutils (after the code has been cleaned up and the proper unit tests introduced). I agree there's a lack of consensus. But I dont' believe that distutils is a strong basis for growth. Distutils may be a reasonable choice of a build tool, (I'm not sure yet), but it's packaging and distribution support is minimal to nonexistent. It is certainly not a good basis as a build tool - it does not handle dependencies for once, and the only thing it really brings is a poor cross-platform implementation to build dynamically loaded libraries, without even a coherent and documented API (you can't introspect something as trivial as compilation flags or where files are built in a cross-platform way, for example). Most of what I'm talking about here speaks to packaging formats, distribution processes, and installation processes. And this isn't new technology. Both debian, rpm, and several other unix technologies have fine systems in operation right now. Sure, they all have weaknesses, but they are much better than easy_install. Not if you don't want/can't spend quite some time to know about each supported platform and the tool details. cheers, David ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
An update on the immediate issue: after discussion elsewhere, it was decided that there were enough other problems with 2.6.3 to warrant a quick release of 2.6.4. Tarek has checked in a change to distutils to unbreak the setuptools currently out in the field. If all goes well, 2.6.4 should be released in a couple of weeks. Thanks, Tarek. -- Ned Deily, n...@acm.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
kiorky wrote: Hi, for the folks using virtualenv-distribute, i forked it to make the last 0.6.3 install instead of 0.6.1. See : http://bitbucket.org/kiorky/virtualenv-distribute/ Install it: easy_install http://distfiles.minitage.org/public/externals/minitage/virtualenv-distribute-1.3.5dev-1.zip So this is a fork of a fork? Nice :-( Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
2009/10/6 Chris Withers ch...@simplistix.co.uk: kiorky wrote: Hi, for the folks using virtualenv-distribute, i forked it to make the last 0.6.3 install instead of 0.6.1. See : http://bitbucket.org/kiorky/virtualenv-distribute/ Install it: easy_install http://distfiles.minitage.org/public/externals/minitage/virtualenv-distribute-1.3.5dev-1.zip So this is a fork of a fork? Nice :-( I think it's a fork of Virtualenv, no? Which uses a fork of distribute. :) We should get virtualenv to support Python3, and I suspect that means we need to get it to support Distribute on Python 2 as well, which probably is a good thing anyway. I do fear trying to understand how it works though, but maybe we can bribe/trick Ian into doing all the work it himself. :-) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
2009/10/6 Lennart Regebro rege...@gmail.com: I think it's a fork of Virtualenv, no? Which uses a fork of distribute. :) I meant that it uses a fork of setuptools, obviously -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
It's just a fork of virtualenv to use distribute. It does not use a fork of distribute but distribute itself ;) Lennart Regebro a écrit : 2009/10/6 Lennart Regebro rege...@gmail.com: I think it's a fork of Virtualenv, no? Which uses a fork of distribute. :) I meant that it uses a fork of setuptools, obviously -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF signature.asc Description: OpenPGP digital signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Anyway, it's released now on pypi as virtualenv-distribute-1.3.4.2. The code is also merged in florian branch and it has been decided that's the main repository. kiorky a écrit : It's just a fork of virtualenv to use distribute. It does not use a fork of distribute but distribute itself ;) Lennart Regebro a écrit : 2009/10/6 Lennart Regebro rege...@gmail.com: I think it's a fork of Virtualenv, no? Which uses a fork of distribute. :) I meant that it uses a fork of setuptools, obviously ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF signature.asc Description: OpenPGP digital signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
2009/10/6 kiorky kio...@cryptelium.net: Anyway, it's released now on pypi as virtualenv-distribute-1.3.4.2. The code is also merged in florian branch and it has been decided that's the main repository. What is the florian branch, and in general, could you provide some more info. Your emails are very cryptic. :) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Ronald Oussoren ronaldousso...@mac.com wrote: Installing distribute is therefore not problematic for most people, if they know that the project exists. The fact that distribute is a seperate project from setuptools can be a problem for people: installing a bugfix release for a software product that we're already using at work is significantly easier than introducing a new software product. For me, it's more a matter of OS X 10.6 already comes with setuptools; how can I mitigate the impact of this buggy unmaintained package on the systems I'm building to deploy on OS X?. Adding distribute to the mix, however good it is, doesn't help; I'm going to use a pure distutils solution. And I can't really install a bugfix release for the Python frameworks in /System; only Apple can do that. Bill ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Hi Lennart, If i read 'virtualenv-distribute 1.3.4.2 on pypi' I can do some googling or even do some Pypi searching for 'virtualenv-distribute'. Thus, the first link found may be [1]. On this link, the second sentence is: The fork was started by Philip Jenvey at http://bitbucket.org/pjenvey/virtualenv-distribute/ and this version by Florian Schulze lives at http://bitbucket.org/fschulze/virtualenv-distribute/ [1] - http://pypi.python.org/pypi/virtualenv-distribute/ Lennart Regebro a écrit : 2009/10/6 kiorky kio...@cryptelium.net: Anyway, it's released now on pypi as virtualenv-distribute-1.3.4.2. The code is also merged in florian branch and it has been decided that's the main repository. What is the florian branch, and in general, could you provide some more info. Your emails are very cryptic. :) -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF signature.asc Description: OpenPGP digital signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Tue, Oct 6, 2009 at 7:00 PM, Bill Janssen jans...@parc.com wrote: For me, it's more a matter of OS X 10.6 already comes with setuptools; how can I mitigate the impact of this buggy unmaintained package on the systems I'm building to deploy on OS X?. Adding distribute to the mix, however good it is, doesn't help; I'm going to use a pure distutils solution. And I can't really install a bugfix release for the Python frameworks in /System; only Apple can do that. I fear the canonical answer to this problem is: Don't use the system Python on Mac OS. It's not particular satisfactory, but as you noticed nobody besides Apple can fix problems in this Python install or update it to newer versions. Apple started to include quite a number of projects, like dateutil, Twisted, NumPy, zope.interface and setuptools to name a few. They will probably update those the next time in Mac OS 10.7 in one or two years. Even today the packages they ship are already outdated and miss bug fixes. So if you want to deploy to Mac OS, I fear the answer is to encourage people to install a good pristine Python version instead. Be that the official GUI installer, Macports, Fink or building your own Python from source. The installers for the Plone application include a full Python build for that reason. Hanno ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
At 09:20 AM 10/6/2009 +0300, Alex Grönholm wrote: P.J. Eby kirjoitti: At 11:53 AM 10/5/2009 +0200, Lennart Regebro wrote: 2009/10/5 Jeff Rush j...@taupro.com: Very unfortunate, as in, it should NOT have happened. And *especially* without any announcement on python.org or mention on the python-committers list of something this major. Well this major... It's a bug fix that breaks a setuptools monkey-patch. Subclassing distutils commands != monkeypatching. If, say, numpy's distutils extensions subclass build_ext and override that method, they could have had the same problem. Same for anybody else's distutils extensions. Setuptools subclasses distutils commands and then replaces the original classes with its own. Example from setuptools/__init__.py: import distutils.core distutils.core.Command = Command Isn't this exactly what monkey patching means? Yes, but that's got nothing to do with the bug that's been being discussed. The same change bit pywin32, and it doesn't use setuptools at all. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
2009/10/6 P.J. Eby p...@telecommunity.com: Yes, but that's got nothing to do with the bug that's been being discussed. The same change bit pywin32, and it doesn't use setuptools at all. True. The problem was a badly documented interface, which meant people used it in one way, but a bug fix assumed another, and *kapow*! The only fixes for that is better documentation and a buildbot, as noticed. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Tue, 06 Oct 2009 10:25:04 -0700, kiorky kio...@cryptelium.net wrote: The fork was started by Philip Jenvey at http://bitbucket.org/pjenvey/virtualenv-distribute/ and this version by Florian Schulze lives at http://bitbucket.org/fschulze/virtualenv-distribute/ [1] - http://pypi.python.org/pypi/virtualenv-distribute/ Just a suggestion: could someone make that /fschulze/virtualenv-distribute site (the current 'home page' of the project) link to the launchpad bug tracker? Up until now, I was not able to track down an issue tracker for this fork .. until you posted the PyPI link. This way one - who stumbled upon the bitbucket site - does not have to pull the source tree and look in docs/index.rst in order to get the URL to the bug tracker. (I had a bug to report a couple of days). -srid ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Oct 6, 2009, at 10:36 AM, Hanno Schlichting wrote: On Tue, Oct 6, 2009 at 7:00 PM, Bill Janssen jans...@parc.com wrote: For me, it's more a matter of OS X 10.6 already comes with setuptools; how can I mitigate the impact of this buggy unmaintained package on the systems I'm building to deploy on OS X?. Adding distribute to the mix, however good it is, doesn't help; I'm going to use a pure distutils solution. And I can't really install a bugfix release for the Python frameworks in /System; only Apple can do that. I fear the canonical answer to this problem is: Don't use the system Python on Mac OS. It's not particular satisfactory, but as you noticed nobody besides Apple can fix problems in this Python install or update it to newer versions. Apple started to include quite a number of projects, like dateutil, Twisted, NumPy, zope.interface and setuptools to name a few. They will probably update those the next time in Mac OS 10.7 in one or two years. Even today the packages they ship are already outdated and miss bug fixes. So if you want to deploy to Mac OS, I fear the answer is to encourage people to install a good pristine Python version instead. Be that the official GUI installer, Macports, Fink or building your own Python from source. The installers for the Plone application include a full Python build for that reason. I believe (but not 100% sure) that all of those projects (setuptools, Twisted, zope.interface) got pulled into site-packages when Apple began shipping iCal stuff. The existance of /usr/bin/easy_install and setuptools is just a side-effect of Apple using an assortment of Python projects to develop iCal. Perhaps we should have got one of those projects that iCal depends upon to put some lightweight need-to- import-it-but-don't-actually-use-it dependencies on VirtualEnv and Buildout. Then those projects would have gotten pulled into Apple's Python and we'd have the tools for easily isolating ourselves from Apple's project's dependencies! Alternatively, someone could work with Apple to get them to deploy their Python apps in a self-contained manner, leaving an Apple Python with a clean site-packages directory, all ready for people to go into and start polluting in a non os-conflicting manner :P (well, technically this directory is clean, it's just overshadowed on sys.path by /System/Library/Frameworks/Python.framework/Versions/2.5/ Extras/lib/python/). They do ship their Ruby apps self-contained. Heck, setuptools provides the same method of working with self- contained library dependencies as Ruby Gems does, and it's not like Apple has demonstrated any of the idiosyncratic setuptools-aversions that exist elsewhere in the Python community. They do after all ship with setuptools and ironically provide a /usr/bin/easy_install even though setuptools also provides entry points as a way of cleanly separating script installation from project installation. C'est la vie! ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Lennart Regebro wrote: 2009/10/5 Jeff Rush j...@taupro.com: Very unfortunate, as in, it should NOT have happened. And *especially* without any announcement on python.org or mention on the python-committers list of something this major. Well this major... It's a bug fix that breaks a setuptools monkey-patch. But yes, it was discovered before release, and maybe it should have been discussed, I'm not on python-dev anymore. The fact that the bugfix (to distutils) was committed by folks who are *alos* pusing a fork to setuptools is what raises the eyebrows here. Note that I am *not* suggesting that anyone acted in bad faith; but the appearance of impropriety can be present where no actual impropriety exists. The only cure is for the conflicted parties to take extreme measures to avoid appearing to act improperly. and even hints of under-the-table intentionality in forcing the community to abandon use of setuptools. There are no such hints anywhere. There are those postig on this thread that the distutils breakage is a good thing for Distribute, because it *forces* folks to switch: I call that a solid hint, myself. Distribute should win because of superior technology not forced migration. That makes no sense. You move to distribute because you have to, because setuptools is buggy and not updated. Until people encounter bugs in setuptools, or need Python 3 support, they are not likely to move to Distribute. There is no other reason than forced migration. But you shouldn't have to move to Distribute becuase the Distrubute developers broke setuptools by changing the Python stdlib (intentionally or not). snip I fail to see how this is a big disaster in any way. Yes, it's not perfect, and yeah, maybe there should have been big warning signs somewhere. But we can NOT leave bugs in Python just because setuptools isn't getting updated. Setuptools has already been a break on Python 3 development, are we gonna lets it be a break on Python 2 bugfixes too? Bugfixes which break backward compatibility in minor relaseses are major fouls, period. As PJE points out, the particular bugfix in question *also* broke other packages (pywin32), which means that it can't be just that setuptools is unmaintained (I'm not supposed to have to re-relase my packages to cope with a patch-level release in Python, period). Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkrMDJcACgkQ+gerLs4ltQ5KxQCg0yhegMhrML5bv7xniRfngG8j FFMAnjmPsBHXZW3sc4/M/qx3N+x1ura2 =2gq3 -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Tres Seaver wrote: Bugfixes which break backward compatibility in minor relaseses are major fouls, period. Sure, but what does backward compatibility even mean for distutils ? Not much, as any non trivial extension needs to use undocumented implementation details. As PJE points out, the particular bugfix in question *also* broke other packages (pywin32), which means that it can't be just that setuptools is unmaintained No, it is a consequence of distutils design and implementation. I think those breakages are unavoidable if you touch distutils besides trivial bug fixes (and the list of trivial things you can do to distutils without breaking anything is tiny). I already had to adapt numpy.distutils numerous times when distutils changed, so I don't think it is fair to point this particular issue as a major foul. It should be accepted that people relying on distutils internals (that is almost anybody using distutils extensions, especially for compiled code) will have to constantly change their code IMHO. cheers, David ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Lennart Regebro wrote: 2009/10/3 Ned Deily n...@acm.org: This is not a good experience for users. Unless I'm missing something (and I hope I am), this issue really can't be hand-waved away. It's unfortunate that this comes in a minor release. Very unfortunate, as in, it should NOT have happened. And *especially* without any announcement on python.org or mention on the python-committers list of something this major. But at the same time we can hardly avoid fixing bugs just because setuptools isn't getting updated at the moment. It's a lose-lose situation. Setuptools is hardly a minor software tool. Considering that setuptools is a major focus and key technology of this group, those making changes to distutils should have tested against setuptools and devised a solution providing backward compatibility, along with deprecation warnings. Lennart and Tarek, I know you at least are familiar with this valuable practice in Zope for years. At the least, those making changes to Distutils should, after testing against Setuptools, widely announced this was coming. It does not reflect positively on the Distribute team and even hints of under-the-table intentionality in forcing the community to abandon use of setuptools. Distribute should win because of superior technology not forced migration. As I see it this will speed up adaptation of Distribute. Word will spread. It's not ideal, but then it's not a perfect world. Distribute is very new and there are many folk who will not be adopting it until it has been out for quite some time. It is a fact of the conservative nature of some development teams. If setuptools were an optional, little-used technology in Python it would not be a big deal but it isn't. It is not appropriate to -force- users to adopt a particular branch of a fork, except perhaps in the move from Python 2.x to 3.x where major changes are anticipated and there was no setuptools for 3.x anyway. Considering that 2.6.3 is messed up in other ways, like displaying the SVN rc1 tag and a broken logging module, and probably will be re-released at 2.6.4 very shortly, perhaps we can get this -at least- into the Python docs and maybe introduce backward compatible hooks into distutils to support setuptools. -Jeff ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
2009/10/5 Jeff Rush j...@taupro.com: Very unfortunate, as in, it should NOT have happened. And *especially* without any announcement on python.org or mention on the python-committers list of something this major. Well this major... It's a bug fix that breaks a setuptools monkey-patch. But yes, it was discovered before release, and maybe it should have been discussed, I'm not on python-dev anymore. and even hints of under-the-table intentionality in forcing the community to abandon use of setuptools. There are no such hints anywhere. Distribute should win because of superior technology not forced migration. That makes no sense. You move to distribute because you have to, because setuptools is buggy and not updated. Until people encounter bugs in setuptools, or need Python 3 support, they are not likely to move to Distribute. There is no other reason than forced migration. Also, there is no win or lose. This is not a competition. Distribute is very new and there are many folk who will not be adopting it until it has been out for quite some time. Nobody will adopt it until they are forced to. This unfortunate bug means people are forced to quicker than expected. I don't think that's an actual problem. It is a fact of the conservative nature of some development teams. Conservative development teams are not likely to either use Subversion 1.6 or Python 2.6.3, so they are not affected by any of the major setuptools problems. I would have expected people starting to get forced to Distribute when major distros where shipping with subversion 1.6. Now it's going to be when they ship with Python 2.6.3 instead. I fail to see how this is a big disaster in any way. Yes, it's not perfect, and yeah, maybe there should have been big warning signs somewhere. But we can NOT leave bugs in Python just because setuptools isn't getting updated. Setuptools has already been a break on Python 3 development, are we gonna lets it be a break on Python 2 bugfixes too? -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Monday, 05 October, 2009, at 11:53AM, Lennart Regebro rege...@gmail.com wrote: 2009/10/5 Jeff Rush j...@taupro.com: Very unfortunate, as in, it should NOT have happened. And *especially* without any announcement on python.org or mention on the python-committers list of something this major. Well this major... It's a bug fix that breaks a setuptools monkey-patch. But yes, it was discovered before release, and maybe it should have been discussed, I'm not on python-dev anymore. I agree with Jeff that this shouldn't have happened, or should at the very least have been documented in the release notes for 2.6.3. I know of several users of Python that have been bitten by this (they installed 2.6.3 and now easy_install doesn't work anymore for them). For beginners this issue is a showstopper that they cannot resolve without help. Distribute is very new and there are many folk who will not be adopting it until it has been out for quite some time. Nobody will adopt it until they are forced to. This unfortunate bug means people are forced to quicker than expected. I don't think that's an actual problem. This is a problem, it means 2.6.3 is not a simple drop-in replacement for 2.6.2 but requires the replacement of another component as well. That can be a problem in organizations with strict configuration management where you cannot install new software without going to lots of red tape (and that might involve lawyers when you install a new package instead of upgrading an existing one). Ronald ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Mon, Oct 5, 2009 at 1:30 PM, Ronald Oussoren ronaldousso...@mac.com wrote: Nobody will adopt it until they are forced to. This unfortunate bug means people are forced to quicker than expected. I don't think that's an actual problem. This is a problem, it means 2.6.3 is not a simple drop-in replacement for 2.6.2 but requires the replacement of another component as well. That can be a problem in organizations with strict configuration management where you cannot install new software without going to lots of red tape (and that might involve lawyers when you install a new package instead of upgrading an existing one). What is your solution ? Setuptools is not part of the standard library and monkey-patch Distutils. Its development has been discontinued for over a year. Everytime Distutils is changed for anything, wether it's a bug fix or not, Setuptools can be broken. Now I realize that some folks wants Distutils to be aware of that and be backward-compatible with Setuptools monkey-patches ? If that so, its means that Distutils can't be fixed and its code can't be changed at all, if Setuptools is not changed in the meantime. We can't force the Setuptools maintainer to do so. Which package is supposely on the top of which one ? A drop in replacement from 2.6.2 to 2.6.3 probably breaks some other softwares out there that are monkey patching some internals of Python, and the way to go is to fix those package. At the end, some folks here sounds like it's Distutils fault if Setuptools is broken, and it can't be simply because Setuptools is not maintained anymore and monkey patches Distutils. Like for the svn 1.6 compat problem we had earlier this year, this problem is a few line changes in Setuptools, it's an 1 hour work. If your company upgrades to Python 2.6.3 it can also upgrade to an hypothetical Setuptools 0.6c..10 ? Tarek -- Tarek Ziadé | http://ziade.org | オープンソースはすごい! | 开源传万世，因有你参与 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
2009/10/5 Ronald Oussoren ronaldousso...@mac.com: This is a problem, it means 2.6.3 is not a simple drop-in replacement for 2.6.2 but requires the replacement of another component as well. That can be a problem in organizations with strict configuration management where you cannot install new software without going to lots of red tape (and that might involve lawyers when you install a new package instead of upgrading an existing one). Sure, but that would have happened sooner or later anyway. Is it really so bad that it happens now instead of say, in half a year? I don't see why it's such a big deal. Sorry, but I don't. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Monday,2009-10-05, at 7:38 , Barry Warsaw wrote: I apologize for my part in this, but moving forward I think that if it's possible to patch and release a setuptools that works with Python 2.6.3 /and/ earlier Python 2.6.x's, then that should happen asap. If that's not possible, then we might need to revert the distutils change in a quick Python 2.6.4. Why not move ahead with both? Then, for example, people who do have Python 2.6.3 installed and get the latest setuptools will be okay, and also people who have setuptools-0.6c9 installed and get the latest Python. There are a lot of people who have constraints on what they they can deploy and when, so it wouldn't hurt to have both fixes available. (Also, one of the fixes might not get done in a timely way.) Thanks, Zooko ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Mon, Oct 5, 2009 at 3:38 PM, Barry Warsaw ba...@python.org wrote: On Oct 5, 2009, at 5:40 AM, Jeff Rush wrote: Very unfortunate, as in, it should NOT have happened. And *especially* without any announcement on python.org or mention on the python-committers list of something this major. [...] Considering that 2.6.3 is messed up in other ways, like displaying the SVN rc1 tag and a broken logging module, and probably will be re-released at 2.6.4 very shortly, perhaps we can get this -at least- into the Python docs and maybe introduce backward compatible hooks into distutils to support setuptools. I apologize for my part in this, but moving forward I think that if it's possible to patch and release a setuptools that works with Python 2.6.3 /and/ earlier Python 2.6.x's, then that should happen asap. If that's not possible, then we might need to revert the distutils change in a quick Python 2.6.4. It's technically possible to fix Setuptools. We did this fix on Distribute, and the patch is 2 lines long. But it's just a matter of having the maintainer doing it. A few months ago we couldn't make him fix and release the bug that made setuptools fail with svn 1.6, and the year before it took several months to get it fixed for svn 1.5 (a one line, not risky change !!!) That's why we have forked and created Distribute, to provide bug fixes. If PJE is not concerned anymore by the maintenance, imho he should let someone that is willing to do it take over the maintenance of his package to fix this (and the other problems). That is not a new problem. Beware that I don't want to run in any new vicious thread here: I had my share of those. So about taking over Setuptools maintenance : 1/ I am not saying it should be me, and I am not saying that I am offended that PJE didn't open the maintenance of setuptools to me. I think he should trust the community and let the maintenance of setuptools be done by all the people that are actively working on the topic. 2/ No, as someone told me in IRC, that's not an evil plan of mine to make people switch to Distribute. This is not in our interest, it's a loss-loss situation. Now I am strongly opposed to revert any bug fix change in Distutils just because it breaks Setuptools, which is unmaintained since a year. We have been struggling over a year with this issue. And we are still struggling because we have to work in a fork to try to provide solutions for the community, with a lot of bootstrapping issues. Now I am astonished that we are talking about reverting changes in Distutils that were done for bugfixes, for a third party package that does monkey patches on Distutils. If this choice wins here, it means that setuptools and the stdlib are tied together, and that the setuptools package should be integrated to the stdlib *immediatly*. Tarek -- Tarek Ziadé | http://ziade.org | オープンソースはすごい! | 开源传万世，因有你参与 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Mon, Oct 5, 2009 at 4:02 PM, Zooko Wilcox-O'Hearn zo...@zooko.com wrote: On Monday,2009-10-05, at 7:38 , Barry Warsaw wrote: I apologize for my part in this, but moving forward I think that if it's possible to patch and release a setuptools that works with Python 2.6.3 /and/ earlier Python 2.6.x's, then that should happen asap. If that's not possible, then we might need to revert the distutils change in a quick Python 2.6.4. Why not move ahead with both? Then, for example, people who do have Python 2.6.3 installed and get the latest setuptools will be okay, and also people who have setuptools-0.6c9 installed and get the latest Python. There are a lot of people who have constraints on what they they can deploy and when, so it wouldn't hurt to have both fixes available. So are you saying that in an environment where you are allowed to install Python 2.6.3, you will not be allowed to install an hypothetical setuptools-0.6c10 (or a Distribute 0.6.3) ? Tarek ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Oct 5, 2009, at 7:44 AM, Lennart Regebro wrote: 2009/10/5 Ronald Oussoren ronaldousso...@mac.com: This is a problem, it means 2.6.3 is not a simple drop-in replacement for 2.6.2 but requires the replacement of another component as well. That can be a problem in organizations with strict configuration management where you cannot install new software without going to lots of red tape (and that might involve lawyers when you install a new package instead of upgrading an existing one). Sure, but that would have happened sooner or later anyway. Is it really so bad that it happens now instead of say, in half a year? I don't see why it's such a big deal. Sorry, but I don't. It's not the happening, it's the happening without a deprecation, warnings, announcements etc. deserved by a change that affects so much of the Python eco-system. S ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Ronald Oussoren wrote: For beginners this issue is a showstopper that they cannot resolve without help. I'm a relative beginner to distutils/setuptools/distribute, but a long time configuration/build/packaging professional. You're mistaken if you think that any of these technologies are suitable for beginners. The state of python package distribution resembles the state of linux packages circa 1995, except that it isn't very well documented at all. Easy_install certainly isn't, and never has been. My first questions about easy_install, (how do I get a list of installed packages? How do I upgrade all installed packages? How do I delete a package using easy_install?) are all unanswerable. I' spent over 20 hours this weekend just reading through documentation trying to figure out what the state of the art is expected to be here and I still don't know much. That's far beyond the investment of a casual user. I'm puzzled that people wonder why I haven't used standard packaging yet when it's pretty clear that standard packaging and distribution doesn't really solve my problems. Python packaging and distribution right now is not for beginners or the faint of heart. --rich - (who came here recently looking for an opportunity to write something akin to stdeb.) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Oct 5, 2009, at 9:38 AM, Barry Warsaw wrote: I apologize for my part in this, but moving forward I think that if it's possible to patch and release a setuptools that works with Python 2.6.3 /and/ earlier Python 2.6.x's, then that should happen asap. If that's not possible, then we might need to revert the distutils change in a quick Python 2.6.4. I thought the whole point of the Distribute project was that we couldn't _get_ a new setuptools release and, so, had to fork. Hopefully, with this motivation, we'll get a new setuptools one more time! and make the transition to Distribute a little more controlled. Fast, but controlled. S ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Oct 5, 2009, at 10:25 AM, K. Richard Pixley wrote: Python packaging and distribution right now is not for beginners or the faint of heart. If we're honest with ourselves, it's not for experienced developers either. Do you really even want to have to /think/ about this stuff? -Barry PGP.sig Description: This is a digitally signed message part ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
2009/10/5 Barry Warsaw ba...@python.org: I apologize for my part in this, but moving forward I think that if it's possible to patch and release a setuptools that works with Python 2.6.3 /and/ earlier Python 2.6.x's, then that should happen asap. PJE seems interested in this, as he asked about a patch, so maybe. If that's not possible, then we might need to revert the distutils change in a quick Python 2.6.4. That would be a big mistake. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
2009/10/5 K. Richard Pixley r...@noir.com: This would be a problem if distribute were in general release. It's not. It's clearly a development branch which is intended to move quickly. No, this is incorrect. The 0.6-branch is not intended to move quickly, it is in bugfix mode. It is moving quickly only because some major bugfixes has been made, but it's not a development branch, that's 0.7, which isn't released. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Mon, Oct 5, 2009 at 4:57 PM, Lennart Regebro rege...@gmail.com wrote: 2009/10/5 Barry Warsaw ba...@python.org: I apologize for my part in this, but moving forward I think that if it's possible to patch and release a setuptools that works with Python 2.6.3 /and/ earlier Python 2.6.x's, then that should happen asap. PJE seems interested in this, as he asked about a patch, so maybe. It was pushed in setuptools bug tracker by Ned yesterday IIRC, with links to the patch that works. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Barry Warsaw wrote: On Oct 5, 2009, at 10:25 AM, K. Richard Pixley wrote: Python packaging and distribution right now is not for beginners or the faint of heart. If we're honest with ourselves, it's not for experienced developers either. Do you really even want to have to /think/ about this stuff? Depends. It's what I do professionally in slightly different arenas, so I think about it anyway. But I take your point. The packaging and distribution system should be simple, powerful, expressive, reliable, and definitive to the point where it's existence fades into the background allowing one to spend one's time and focus of attention on less easily solved problems. --rich ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Lennart Regebro wrote: 2009/10/5 K. Richard Pixley r...@noir.com: This would be a problem if distribute were in general release. It's not. It's clearly a development branch which is intended to move quickly. No, this is incorrect. The 0.6-branch is not intended to move quickly, it is in bugfix mode. It is moving quickly only because some major bugfixes has been made, but it's not a development branch, that's 0.7, which isn't released. I'm recent to python packaging and distribution, so let me see if I've put this together right from my reading of the various web pages involved over the weekend. Distutils is currently part of the standard python library. As such, it's released with python, (the reference implementation, anyway). Distutils is currently capable of producing only source archives of packages. While it's capable of producing built archives, those archives are machine specific, nonrelocatable, untrackable, and have no standard method for distribution nor installation nor tracking. Setuptools was a third party addition to, (and partial replacement of), distutils because distutils wasn't suitably usable nor was it moving fast enough. However, since setuptools was initiated, many of the major features of setuptools have since been folded back into distutils, making setuptools partially redundant and partly colliding. Setuptools provides the ability to produce machine independent built archives and a standard method for installing them, (although not for tracking or removing them). And the setuptools approach to installation, easy_install, doesn't play nice with the native installers on systems that have them like rpm, debian, etc. However, setuptools has fallen into disrepair and so distribute has been created, as a friendly branch off a third party tool which in turn was a form of branch off of distutils. And within distribute, there are two lines of development, the 0.7 line, which is intended to replace... I'm confused. Does it replace setuptools or distutils? And then there's the 0.6 branch, which is a branch off 0.7 which is a branch of setuptools which is a branch of distutils which is under recent active development and yet it also expected to be stable, as much as such a term can be applied to a third party branch off a third party of a colliding replacement with a standard facility. Is that about right? If I'm anywhere near right, then I can't really imagine what state you intend for the 0.6 branch if not development. --rich ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
2009/10/5 K. Richard Pixley r...@noir.com: Is that about right? Nope. 0.6 is a fork of setuptools, providing bugfixes (and also 3.1 support). It's completely backwards compatible with setuptools. 0.7 is a development branch, which aims to refactor setuptools into something or (rather several somethings) that is simpler and easier to maintain. It will not be backwards compatible. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
K. Richard Pixley wrote: Ronald Oussoren wrote: For beginners this issue is a showstopper that they cannot resolve without help. I'm a relative beginner to distutils/setuptools/distribute, but a long time configuration/build/packaging professional. You're mistaken if you think that any of these technologies are suitable for beginners. The state of python package distribution resembles the state of linux packages circa 1995, except that it isn't very well documented at all. As a general question, is there any planned project to improve the state of distutils or replace it? It appears to be one of the weakest parts of the Python system and needs replacing with something much cleaner, better documented and more powerful. Even making something like cmake the standard would help a lot. Jeremy -- jer...@jeremysanders.net http://www.jeremysanders.net/ ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Jeremy Sanders kirjoitti: K. Richard Pixley wrote: Ronald Oussoren wrote: For beginners this issue is a showstopper that they cannot resolve without help. I'm a relative beginner to distutils/setuptools/distribute, but a long time configuration/build/packaging professional. You're mistaken if you think that any of these technologies are suitable for beginners. The state of python package distribution resembles the state of linux packages circa 1995, except that it isn't very well documented at all. As a general question, is there any planned project to improve the state of distutils or replace it? It appears to be one of the weakest parts of the Python system and needs replacing with something much cleaner, better documented and more powerful. Even making something like cmake the standard would help a lot. There is a lack of consensus regarding how exactly they should work. If we are having this much trouble deciding how a third party tool should work, it is certainly not going to be merged into distutils until those issues have been resolved. Distutils is what houses (or should) the parts we all agree on. That said, I think that plenty of setuptools/distribute functionality should be moved to distutils (after the code has been cleaned up and the proper unit tests introduced). Jeremy ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Monday,2009-10-05, at 8:11 , Tarek Ziadé wrote: So are you saying that in an environment where you are allowed to install Python 2.6.3, you will not be allowed to install an hypothetical setuptools-0.6c10 (or a Distribute 0.6.3) ? Yes, situations like that can come up. For example, I guess the packaging of my own Tahoe-LAFS project is a case in point. The current requirements for Tahoe-LAFS are Python = 2.4.2 and 3.0, and the source distribution of Tahoe-LAFS comes with its own bundled copy of setuptools. We haven't finished our qualification of Distribute so we're not ready to switch our build system to Distribute. Setuptools-0.6c10 is hypothetical at this point. What do we do? We could tighten the Python version to = 2.4.2 and = 2.6.2. We could develop and test a patch to our bundled copy of setuptools to work-around this problem. We could ask the open source volunteer who is testing Distribute for us to hurry up. We could ask PJE to hurry up and release a new version of setuptools. Perhaps we will end up doing more than one of these things. Many of our users won't even be able to diagnose the problem if they upgrade from Python 2.6.2 to 2.6.3 and the install breaks. They won't know what is wrong or how to work around it, other than by writing to our mailing list asking for help. The faster this gets fixed and the more lenient Python 2.6.x is with respect to what versions of setuptools/Distribute it requires and the more lenient setuptools and Distribute are with respect to what version of Python they require, the better for everyone. Thank you kindly for your attention. Regards, Zooko ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
I'm sorry to follow-up to my own post, but I realized that I didn't make something clear: the current Tahoe-LAFS source distribution comes with its own copy of setuptools, so even if PJE releases a new version of setuptools or if we patch that copy to work-around this problem, we're going to have to re-qualify the combination of the new version on our buildbot and make a new stable release of Tahoe-LAFS (something that typically only happens every 3 or 4 months) before end users will be able to install Tahoe-LAFS on Python 2.6.3. I don't want to point fingers, because that goes nowhere. Lots of people could have written their software or managed their processes differently in order to avoid this situation, including me. However, I do want to emphasize that this is a serious problem. Backwards compatibility on minor releases such as from 2.6.2 to 2.6.3 is a huge concern for a lot of people. If something like this breaks -- regardless of whose fault(s) it is -- then it reduces people's trust in the stability of stable updates of their infrastructural software. I'm sorry to say that this event has already made me more hesitant to jump from setuptools to Distribute, just because some of the maintainers of Distribute have posted saying that they don't think this kind of thing is such a big deal. I prefer to use packaging tools which are stable. To achieve that kind of stability sometimes requires basically taking responsibility for other people's decisions, such as saying Well, setuptools-0.6c9 monkey-patches distutils. We think that's a bad idea and we wish it didn't do that. But, we know that a lot of other people out there are relying on the combination of setuptools-0.6c9 and Python 2.6.x, so we're going to go the extra mile to make sure that those people don't experience disruptions.. Please take this in the constructive spirit that is intended. We're on the same team. I've already contributed a few patches and a lot of bug reports to setuptools, Distribute, and Python, and I'd like to contribute more in the future. Having a policy of actively working to maintain stability across stable updates will help everyone. Regards, Zooko ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Alex Grönholm wrote: There is a lack of consensus regarding how exactly they should work. If we are having this much trouble deciding how a third party tool should work, it is certainly not going to be merged into distutils until those issues have been resolved. Distutils is what houses (or should) the parts we all agree on. That said, I think that plenty of setuptools/distribute functionality should be moved to distutils (after the code has been cleaned up and the proper unit tests introduced). I agree there's a lack of consensus. But I dont' believe that distutils is a strong basis for growth. Distutils may be a reasonable choice of a build tool, (I'm not sure yet), but it's packaging and distribution support is minimal to nonexistent. I think what's needed here isn't a consensus, (which we'll never get), but rather a generational solution based not on available technology, but rather on state of the art requirements. For example, distutils says nothing about host packaging systems nor distribution. The distutils doc doesn't really even broach the topic of impure python packages or distribution. I don't know, for instance, whether C extensions are expected to be contained in a collection of different tar.gz archives or if there's expected to be one fat one which contains binary C extensions for a collection of different operating systems. The new system needs to support mass updates of all installed packages, querying of installed packages, removal of installed packages, recursive instances of the packaging database, (perhaps with something like virtualenv), as well as all of the things that easy_install supports now, and most of the features that debian/rpm/etc support now including virtual packages, dependencies, suggestions, conflicts, and both command line and gui installers. It should also be capable of at least playing nice with the host packaging system, (eg, rpm, deb, ipk, etc), or perhaps of completely excusing itself from the distribution process, (other than supporting the native distribution process). If necessary, we can create extra repositories for those systems such that python packages can be distributed through the native packaging system, even though we're managing our own release processes. I know that debian/ubuntu/rpm/easy_install all allow for this as it's very common for enterprises to use such a solution for distributing their own packages in-house. The new system should also support cross compilation to the extent that such is needed for C extensions and to the extent that the native packaging system supports. (Eg, debian doesn't, really but bitbake/OE virtually requires it). Most of what I'm talking about here speaks to packaging formats, distribution processes, and installation processes. And this isn't new technology. Both debian, rpm, and several other unix technologies have fine systems in operation right now. Sure, they all have weaknesses, but they are much better than easy_install. --rich ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Mon, Oct 5, 2009 at 1:33 PM, Ned Deily n...@acm.org wrote: I've opened an issue of the main Python issue tracker outlining the problem, primarily for the benefit of affected users who search the tracker: http://bugs.python.org/issue7064 If I understand the comments on this ticket correctly, Tarek has changed distutils in a way so the last setuptools release continues to work, correct? So based on the current state of Python 2.6.4 will work again with an unmodified setuptools 0.6c9? Hanno ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
At 07:25 AM 10/5/2009 -0700, K. Richard Pixley wrote: How do I delete a package using easy_install? http://peak.telecommunity.com/DevCenter/EasyInstall#uninstalling-packages ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
At 04:57 PM 10/5/2009 +0200, Lennart Regebro wrote: 2009/10/5 Barry Warsaw ba...@python.org: I apologize for my part in this, but moving forward I think that if it's possible to patch and release a setuptools that works with Python 2.6.3 /and/ earlier Python 2.6.x's, then that should happen asap. PJE seems interested in this, as he asked about a patch, so maybe. I expect to have a small amount of time later this week to work on setuptools. No guarantees, but this is certainly one of the bugs on my short list for doing something with. If that's not possible, then we might need to revert the distutils change in a quick Python 2.6.4. That would be a big mistake. Actually, I think the mistake here is where non-bugfix code was ported from 3.x back to the 2.6.x maintenance branch. I'm particularly concerned about the .compiler / ._compiler change, as it seems to me to be responsible for breaking mingw32 compilation on Windows, but I haven't had time to investigate thoroughly. (It actually looks like a problem that might be in Python 3 as well.) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
At 06:53 PM 10/5/2009 +0200, Lennart Regebro wrote: Possibly if you somehow think it's the Distribute teams fault that a bugfix in Python ended up breaking setuptools. If it would have been better not to fix that bug, then the blame reasonably goes to the Python core developers, not the Distribute team. In this case, though, the Python core developer is also the Distribute lead. (i.e., it was Tarek who made the changes to the distutils.) So it's a bit understandable that some people might wonder if there was a conflict of interest. I don't personally think that's the case; it's pretty much inevitable that the distutils making progress means other things will break. But it's easy to see how others might take the situation another way, or treat it as an example of Distribute policy towards backward compatibility, or of what kind of breakage is considered acceptable in a dot release. It would be good to bear in mind that extending the distutils (or setuptools) is *not* monkeypatching; both libraries provide explicit assurance that subclassing is in fact allowed. And there's nothing all that special about setuptools' subclassing of build_ext; in fact, if you look back in the archives here, other people have done equivalent subclassing to support dynamic library building. I haven't checked their code, but there is a strong possibility that it would also fail in the same way. This is not really about monkeypatching, or about special support for setuptools. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
P.J. Eby wrote: At 07:25 AM 10/5/2009 -0700, K. Richard Pixley wrote: How do I delete a package using easy_install? http://peak.telecommunity.com/DevCenter/EasyInstall#uninstalling-packages That doesn't remove a package. It simply removes the package from the search path by one method in hopes that no further instances of it will be loaded. To actually remove it, you have to know the format of the library, the package formats, local library storage conventions, and you need to be an expert user of distutils, setuptools, buildout, etc, in order to determine the content of the package itself in order to remove it manually. And even then you'd have to manually search your entire system for any packages that might still depend on this package lest you break them too. That's far beyond the scope of expectation for a casual package maintainer. That's more akin to current macosx standards, (install only), than debian or rpm, (install, update, query, remove, etc). I suppose it's not such a bad problem if you reload your OS frequently. Or solely work with virtualenv or the like so that you can easily discard and rebuild an environment whenever you need one. Of course, that doesn't really help with the task of keeping multiple servers current or rolling internal software out to an enterprise. --rich ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
At 07:53 PM 10/5/2009 +0200, Hanno Schlichting wrote: If I understand the comments on this ticket correctly, Tarek has changed distutils in a way so the last setuptools release continues to work, correct? Yes. And a very nice fix, done quite quickly. Thank you Tarek. So based on the current state of Python 2.6.4 will work again with an unmodified setuptools 0.6c9? AFAICT, that is correct. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
I'm struggling to articulate something here. When the maintainer of the stable branch of a platform that I rely on says The fact that upgrading to our recent stable release will break this critical functionality is so-and-so's fault, not ours. this reduces my confidence in that maintainer. Not because he's wrong! Maybe it *is* so-and-so's fault. But what I'm looking for in the maintainer of a stable platform is someone who says Maybe this wasn't our fault, but here are the steps we're taking to get you back on your feet as soon as possible.. Regards, Zooko ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Mon, Oct 5, 2009 at 8:38 PM, P.J. Eby p...@telecommunity.com wrote: At 07:53 PM 10/5/2009 +0200, Hanno Schlichting wrote: If I understand the comments on this ticket correctly, Tarek has changed distutils in a way so the last setuptools release continues to work, correct? Yes. And a very nice fix, done quite quickly. Thank you Tarek. Wonderful. This seems to be the right approach to the current problem for me. Thank you Tarek indeed! So based on the current state of Python 2.6.4 will work again with an unmodified setuptools 0.6c9? AFAICT, that is correct. Good. So we can hopefully end this thread and move on to something more productive. If anyone wants to step up and provide help in testing pre-releases of Python, so we can avoid similar situation in the future, that would be most welcome. Hanno ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
At 11:29 AM 10/5/2009 -0700, K. Richard Pixley wrote: P.J. Eby wrote: At 07:25 AM 10/5/2009 -0700, K. Richard Pixley wrote: How do I delete a package using easy_install? http://peak.telecommunity.com/DevCenter/EasyInstall#uninstalling-packages That doesn't remove a package. It simply removes the package from the search path by one method in hopes that no further instances of it will be loaded. To actually remove it, you have to know the format of the library, the package formats, local library storage conventions, and you need to be an expert user of distutils, setuptools, buildout, etc, in order to determine the content of the package itself in order to remove it manually. As it says at the above link: After you've done this, you can safely delete the .egg files or directories, along with any scripts you wish to remove. What it doesn't mention (but which should be apparent if you actually run the command) is that it will output the locations of the .egg files or directories and scripts in question, allowing you to copy and paste them to 'rm' or 'del' commands. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
2009/10/5 K. Richard Pixley r...@noir.com: I'm recent to python packaging and distribution, so let me see if I've put this together right from my reading of the various web pages involved over the weekend. Distutils is currently part of the standard python library. As such, it's released with python, (the reference implementation, anyway). Distutils is currently capable of producing only source archives of packages. While it's capable of producing built archives, those archives are machine specific, nonrelocatable, untrackable, and have no standard method for distribution nor installation nor tracking. Setuptools was a third party addition to, (and partial replacement of), distutils because distutils wasn't suitably usable nor was it moving fast enough. However, since setuptools was initiated, many of the major features of setuptools have since been folded back into distutils, making setuptools partially redundant and partly colliding. Setuptools provides the ability to produce machine independent built archives and a standard method for installing them, (although not for tracking or removing them). And the setuptools approach to installation, easy_install, doesn't play nice with the native installers on systems that have them like rpm, debian, etc. However, setuptools has fallen into disrepair and so distribute has been created, as a friendly branch off a third party tool which in turn was a form of branch off of distutils. And within distribute, there are two lines of development, the 0.7 line, which is intended to replace... I'm confused. Does it replace setuptools or distutils? And then there's the 0.6 branch, which is a branch off 0.7 which is a branch of setuptools which is a branch of distutils which is under recent active development and yet it also expected to be stable, as much as such a term can be applied to a third party branch off a third party of a colliding replacement with a standard facility. Is that about right? If I'm anywhere near right, then I can't really imagine what state you intend for the 0.6 branch if not development. I believe that the Distribute 0.6 branch is a stable continuation of the setuptools (0.6) stable branch. Distribute 0.6 is intended to include maintenance-only stable fixes (on the basis that setuptools no longer provides even that minimal level of progress - something on which I won't comment for fear of starting another long thread). Distribute 0.7 is the development branch, looking at new features such as Python 3 support. The only other point you missed is that there is not universal approval of setuptools as a solution to the packaging problem. Take-up of setuptools (and eggs, and easy install, and the various other aspects of the setuptools ecosystem) is variable, with (as far as I see it) strong support from the web development community, and mixed reception elsewhere in the Python community. Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Mon, 05 Oct 2009 11:21:28 -0700, P.J. Eby p...@telecommunity.com wrote: And there's nothing all that special about setuptools' subclassing of build_ext; in fact, if you look back in the archives here, other people have done equivalent subclassing to support dynamic library building. I haven't checked their code, but there is a strong possibility that it would also fail in the same way. Correct. pywin32-212 broke in similar way - http://bugs.python.org/issue7020 - which issue was fixed in pywin32 trunk. And Tarek commented on the incompleteness of the `get_ext_filename` API: TAREK: [...] But this API, even if its doctest doesn't make it clear (I will change it to make it clearer) is used for both namespaced names and non namespaced names by the community. Personally, I prefer that setuptools is fixed (in accordance with the clarified API behavior). Yet, 2.6.3 currently has a critical bug open with the logging module that may warrant a quick 2.6.4 release. -srid ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Zooko Wilcox-O'Hearn zo...@zooko.com wrote: I'm struggling to articulate something here. When the maintainer of the stable branch of a platform that I rely on says The fact that upgrading to our recent stable release will break this critical functionality is so-and-so's fault, not ours. this reduces my confidence in that maintainer. Not because he's wrong! Maybe it *is* so-and-so's fault. But what I'm looking for in the maintainer of a stable platform is someone who says Maybe this wasn't our fault, but here are the steps we're taking to get you back on your feet as soon as possible.. Zooko, I've struggled with this over the last year, in integrating a dozen fairly complicated third-party Python extensions for my system. I've come to the conclusion that the problem is setuptools, and I'm trying hard to remove it entirely from my system. I have no problem with the .egg format or the basic idea, or even the implementation, which I think is pretty nice. It's the structure of the setuptools project that gives me pause. There seems to be one developer, and he seems to be too busy to fix the well-known bugs (like having easy_install ruin the sys.path settings by putting stuff on it in the wrong place -- is that one fixed yet?). In addition, I think the mere existence of setuptools stifles progress on distutils, which is where all the clever tricks of setuptools should properly appear. This would let the whole community of Python developers work on the codebase. I'd like to see a flag on PyPI marking whether the package relies on setuptools, in which case I'll avoid it, or voluntarily entangles itself with setuptools if present (as the only known way to create eggs), or is setuptools-free (my preferred configuration). Frankly, I'd also recommend putting up a warning to developers on PyPI noting these problems with setuptools. Bill ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
2009/10/5 Jeremy Sanders jer...@jeremysanders.net: As a general question, is there any planned project to improve the state of distutils or replace it? It appears to be one of the weakest parts of the Python system and needs replacing with something much cleaner, better documented and more powerful. Tarek's working on enhancements to distutils. The current breakage of setuptools demonstrates why this is a slow, painful, process :-( Even making something like cmake the standard would help a lot. There's no plan for anything like that. I'll leave it to you to imagine the cries of horror from the community when such a radical change meant that easy_install no longer worked, and there was no prospect of fixing it :-) Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Bill Janssen kirjoitti: Zooko Wilcox-O'Hearn zo...@zooko.com wrote: I'm struggling to articulate something here. When the maintainer of the stable branch of a platform that I rely on says The fact that upgrading to our recent stable release will break this critical functionality is so-and-so's fault, not ours. this reduces my confidence in that maintainer. Not because he's wrong! Maybe it *is* so-and-so's fault. But what I'm looking for in the maintainer of a stable platform is someone who says Maybe this wasn't our fault, but here are the steps we're taking to get you back on your feet as soon as possible.. Zooko, I've struggled with this over the last year, in integrating a dozen fairly complicated third-party Python extensions for my system. I've come to the conclusion that the problem is setuptools, and I'm trying hard to remove it entirely from my system. I have no problem with the .egg format or the basic idea, or even the implementation, which I think is pretty nice. It's the structure of the setuptools project that gives me pause. There seems to be one developer, and he seems to be too busy to fix the well-known bugs (like having easy_install ruin the sys.path settings by putting stuff on it in the wrong place -- is that one fixed yet?). In addition, I think the mere existence of setuptools stifles progress on distutils, which is where all the clever tricks of setuptools should properly appear. This would let the whole community of Python developers work on the codebase. I'd like to see a flag on PyPI marking whether the package relies on setuptools, in which case I'll avoid it, or voluntarily entangles itself with setuptools if present (as the only known way to create eggs), or is setuptools-free (my preferred configuration). Frankly, I'd also recommend putting up a warning to developers on PyPI noting these problems with setuptools. Does your bug still exist in Distribute? If so, please report it at http://bitbucket.org/tarek/distribute/ (assuming that bitbucket is operational, which it currently isn't) Bill ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Alex Grönholm alex.gronh...@nextday.fi wrote: Does your bug still exist in Distribute? If so, please report it at http://bitbucket.org/tarek/distribute/ (assuming that bitbucket is operational, which it currently isn't) Sorry, Alex, I don't know about Distribute, don't (particularly) care. If you care, test for it and report it if it's there. It's bug 53 in the setuptools bug reporter. Bill ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Bill Janssen kirjoitti: Alex Grönholm alex.gronh...@nextday.fi wrote: Does your bug still exist in Distribute? If so, please report it at http://bitbucket.org/tarek/distribute/ (assuming that bitbucket is operational, which it currently isn't) Sorry, Alex, I don't know about Distribute, don't (particularly) care. If you care, test for it and report it if it's there. It's bug 53 in the setuptools bug reporter. Bill If you are seriously expecting setuptools to be fixed, I can only assume you haven't been following the conversation on this list. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On 5 Oct, 2009, at 16:25, K. Richard Pixley wrote: Ronald Oussoren wrote: For beginners this issue is a showstopper that they cannot resolve without help. I'm a relative beginner to distutils/setuptools/distribute, but a long time configuration/build/packaging professional. You're mistaken if you think that any of these technologies are suitable for beginners. The state of python package distribution resembles the state of linux packages circa 1995, except that it isn't very well documented at all. I didn't say that distutils is suitable for beginners, but beginners to use them and are confused when they stop working. Easy_install certainly isn't, and never has been. My first questions about easy_install, (how do I get a list of installed packages? How do I upgrade all installed packages? How do I delete a package using easy_install?) are all unanswerable. All of these are not yet implemented. PJE has mentioned that he wants to work on this in for setuptools 0.7, but development of setuptools has completely stalled. The distribute package was created because of this, and development is moving forward again. Ronald ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On 5 Oct, 2009, at 16:37, K. Richard Pixley wrote: Ronald Oussoren wrote: This is a problem, it means 2.6.3 is not a simple drop-in replacement for 2.6.2 but requires the replacement of another component as well. That can be a problem in organizations with strict configuration management where you cannot install new software without going to lots of red tape (and that might involve lawyers when you install a new package instead of upgrading an existing one). This would be a problem if distribute were in general release. It's not. It's clearly a development branch which is intended to move quickly. People using distribute are taking development, pre-alpha kinds of risk and that has been made pretty clear already. AFAIK distribute 0.6 is a stable release, basicly setuptools 0.6c9 + bugfixes + py3k support. Installing distribute is therefore not problematic for most people, if they know that the project exists. The fact that distribute is a seperate project from setuptools can be a problem for people: installing a bugfix release for a software product that we're already using at work is significantly easier than introducing a new software product. Ronald ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
Hi, for the folks using virtualenv-distribute, i forked it to make the last 0.6.3 install instead of 0.6.1. See : http://bitbucket.org/kiorky/virtualenv-distribute/ Install it: easy_install http://distfiles.minitage.org/public/externals/minitage/virtualenv-distribute-1.3.5dev-1.zip Ned Deily a écrit : I'm afraid there is going to be a small deluge of very confused users who will end up needing to install Distribute but only when they eventually figure out why some packages with C extensions mysteriously no longer install after they upgrade to python 2.6.3. For example, following the package instructions to use setuptools easy_install, an easy_install lxml fails with 2.6.3 with the very cryptic: File build/bdist.macosx-10.3-fat/egg/setuptools/command/build_ext.py, line 85, in get_ext_filename KeyError: 'etree' http://stackoverflow.com/questions/1512530/cant-install-lxml-python-2-6-3 -osx-10-6-snow-leopard Appscript is another package that fails similarly. There must be others. I don't know what's to be done about this awkward incompatibility now that 2.6.3 is out the door other than perhaps making sure maintainers of affected packages are aware of what their users are running into and can update their documentation. There is one small thing that may help some users who find their way to http://pypi.python.org/pypi/distribute. OS X, for one, does not ship with wget however it does have curl. Suggest adding another example: $ curl -O http://nightly.ziade.org/distribute_setup.py $ python distribute_setup.py -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF signature.asc Description: OpenPGP digital signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
At 03:49 PM 10/3/2009 +0200, Tarek Ziadé wrote: Notice that this has been fixed in Ubuntu already with a patched version of setuptools Is the patch or an equivalent already in the setuptools tracker? And if not, can someone please post it there? Thanks. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
2009/10/3 Ned Deily n...@acm.org: I'm afraid there is going to be a small deluge of very confused users who will end up needing to install Distribute but only when they eventually figure out why some packages with C extensions mysteriously no longer install after they upgrade to python 2.6.3. For example, following the package instructions to use setuptools easy_install, an easy_install lxml fails with 2.6.3 with the very cryptic: File build/bdist.macosx-10.3-fat/egg/setuptools/command/build_ext.py, line 85, in get_ext_filename KeyError: 'etree' http://stackoverflow.com/questions/1512530/cant-install-lxml-python-2-6-3 -osx-10-6-snow-leopard Appscript is another package that fails similarly. There must be others. I can confirm this. It seems to be packages that use c-extensions. zope.interface also breaks. $ curl -O http://nightly.ziade.org/distribute_setup.py $ python distribute_setup.py Distribute doesn't break, though, so sudo easy_install Distribute seems to be enough. :-) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Sat, Oct 3, 2009 at 2:15 PM, Lennart Regebro rege...@gmail.com wrote: 2009/10/3 Ned Deily n...@acm.org: I'm afraid there is going to be a small deluge of very confused users who will end up needing to install Distribute but only when they eventually figure out why some packages with C extensions mysteriously no longer install after they upgrade to python 2.6.3. For example, following the package instructions to use setuptools easy_install, an easy_install lxml fails with 2.6.3 with the very cryptic: File build/bdist.macosx-10.3-fat/egg/setuptools/command/build_ext.py, line 85, in get_ext_filename KeyError: 'etree' http://stackoverflow.com/questions/1512530/cant-install-lxml-python-2-6-3 -osx-10-6-snow-leopard Appscript is another package that fails similarly. There must be others. I can confirm this. It seems to be packages that use c-extensions. zope.interface also breaks. Yes Setuptools makes some assumptions on how distutils APIs are called and in which order. Once bitbucket is up, you can look at the details on the fix I did, it's fairly simple. Notice that this has been fixed in Ubuntu already with a patched version of setuptools, but the next probable step is to use Distribute there and in debian. $ curl -O http://nightly.ziade.org/distribute_setup.py $ python distribute_setup.py Distribute doesn't break, though, so sudo easy_install Distribute seems to be enough. :-) Yes that's the simplest way to fix the problem. Notice that setuptools patches distutils in other places, and it will not work with Python 2.7 and 3.2 where Distribute does. Cheers Tarek -- Tarek Ziadé | http://ziade.org | オープンソースはすごい! | 开源传万世，因有你参与 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
In article 94bdd2610910030649r431a5638y7c8b5332934f...@mail.gmail.com, Tarek Ziad? ziade.ta...@gmail.com wrote: On Sat, Oct 3, 2009 at 2:15 PM, Lennart Regebro rege...@gmail.com wrote: 2009/10/3 Ned Deily n...@acm.org: I'm afraid there is going to be a small deluge of very confused users who will end up needing to install Distribute but only when they eventually figure out why some packages with C extensions mysteriously no longer install after they upgrade to python 2.6.3. For example, following the package instructions to use setuptools easy_install, an easy_install lxml fails with 2.6.3 with the very cryptic: File build/bdist.macosx-10.3-fat/egg/setuptools/command/build_ext.py, line 85, in get_ext_filename KeyError: 'etree' http://stackoverflow.com/questions/1512530/cant-install-lxml-python-2-6-3 -osx-10-6-snow-leopard Appscript is another package that fails similarly. There must be others. I can confirm this. It seems to be packages that use c-extensions. zope.interface also breaks. Yes Setuptools makes some assumptions on how distutils APIs are called and in which order. Once bitbucket is up, you can look at the details on the fix I did, it's fairly simple. Notice that this has been fixed in Ubuntu already with a patched version of setuptools, but the next probable step is to use Distribute there and in debian. That's fine but how does this help users on OS X, for example, when the python 2.6.3 they are using is supplied only by the python.org installers? And the maintainers of the packages that can't be installed? $ curl -O http://nightly.ziade.org/distribute_setup.py $ python distribute_setup.py Distribute doesn't break, though, so sudo easy_install Distribute seems to be enough. :-) Yes that's the simplest way to fix the problem. But that's assuming they have a working easy_install installed for *that* version (2.6.3) in the first place. For instance, the OS X installer does not include setuptools or Distribute. I'm concerned about users who just want to install some third-party package for the first time. They try to follow the recipe for bootstrapping Distribute and fail right at the first step because they don't have wget on their system (for example, OS X) and they don't know what it is. Getting it right for beginners is one of the reasons why easy_install was developed in the first place. Notice that setuptools patches distutils in other places, and it will not work with Python 2.7 and 3.2 where Distribute does. That's fine, too, and there are lessons to be learned here, I think, that should influence 2.7 and 3.2. But the big problem is distutils in 2.6.3. This is not trying to debate the Distribute fork; the target audience for easy_install doesn't care one way or the other. More importantly, they haven't a clue that there is something called Distribute and they don't know what setuptools is. Like it or not, easy_install / setuptools is a major customer of distutils and somehow it got broken for 2.6.3, a point release. From the end user and the 3rd-party packagers' points of view: the 3rd-party package didn't change, setuptools didn't change (obviously!), but now the package won't install and, worse, fails cryptically. So the problem must be Python - so they think and with some justification, it seems. Once they get Distribute installed, everything should be fine but how do they know they are supposed to install Distribute? This is not a good experience for users. Unless I'm missing something (and I hope I am), this issue really can't be hand-waved away. -- Ned Deily, n...@acm.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Oct 3, 2009, at 1:00 PM, Ned Deily wrote: This is not a good experience for users. Unless I'm missing something (and I hope I am), this issue really can't be hand-waved away. What would you suggest? S ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
2009/10/3 Ned Deily n...@acm.org: This is not a good experience for users. Unless I'm missing something (and I hope I am), this issue really can't be hand-waved away. It's unfortunate that this comes in a minor release. But at the same time we can hardly avoid fixing bugs just because setuptools isn't getting updated at the moment. It's a lose-lose situation. As I see it this will speed up adaptation of Distribute. Word will spread. It's not ideal, but then it's not a perfect world. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Sat, Oct 3, 2009 at 7:00 PM, Ned Deily n...@acm.org wrote: This is not a good experience for users. Unless I'm missing something (and I hope I am), this issue really can't be hand-waved away. Make sure to understand that the way setuptools patches distutils makes it very sensible to any change made in distutils, even backward compatibles ones like in the 2.6 branch. But I won't freeze distutils work because of that, and distutils is not to be blamed if setuptools is broken, neither Python. Especially for a project that has not been maintained for over a year... So it's not a good experience for the users that's true, and we can try to enhance the Distribute documentation to help them as you suggested, and we are already helping because we've forked to try to provide some solutions. But there's nothing else I can think of to help people. My advice would be that the projects that use setuptools just switch asap on distribute, which is actively maintained. -- Ned Deily, ...@acm.org ___ Distutils-SIG maillist - distutils-...@python.org http://mail.python.org/mailman/listinfo/distutils-sig -- Tarek Ziadé | http://ziade.org | オープンソースはすごい! | 开源传万世，因有你参与 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Sat, Oct 3, 2009 at 10:09 PM, Tarek Ziadé ziade.ta...@gmail.com wrote: ...makes it very sensible to any change made in distutils, even backward compatibles ones like in the 2.6 branch s/backward compatibles/ bug fixes/ ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Oct 3, 2009, at 4:08 PM, Lennart Regebro wrote: 2009/10/3 Ned Deily n...@acm.org: This is not a good experience for users. Unless I'm missing something (and I hope I am), this issue really can't be hand-waved away. How about some sort of an announcement/warning on the setuptools site itself? I know the code's not going to get updated but how about a simple warning and suggestion to move on to Distribute? S ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
In article 94bdd2610910031309w61d72dcdo8faab4964bf67...@mail.gmail.com, Tarek Ziadé ziade.ta...@gmail.com wrote: On Sat, Oct 3, 2009 at 7:00 PM, Ned Deily n...@acm.org wrote: This is not a good experience for users. Unless I'm missing something (and I hope I am), this issue really can't be hand-waved away. Make sure to understand that the way setuptools patches distutils makes it very sensible to any change made in distutils, even backward compatibles ones like in the 2.6 branch. But I won't freeze distutils work because of that, and distutils is not to be blamed if setuptools is broken, neither Python. I understand very well the issues that lead to the establishment of the Distribute project and I think it is great that you have taken on this - at times - pretty thankless effort. So thank you for doing so. I'm not trying to point fingers here; if I were, I'd start with me not recognizing this issue before 2.6.3 went out the door. I'm simply trying to put myself in the place of the naive user, the target audience of easy_install. And users, naive or otherwise, don't know or care who is to blame, they'll just know that something about using Python 2.6.3 is broken. So it's not a good experience for the users that's true, and we can try to enhance the Distribute documentation to help them as you suggested, and we are already helping because we've forked to try to provide some solutions. But there's nothing else I can think of to help people. My advice would be that the projects that use setuptools just switch asap on distribute, which is actively maintained. That's fine but they're not going to know about Distribute unless they stumble across discussions like this. And it's not their usage of setuptools that is the problem here. I think most of them would view this (rightly or wrongly) as: Without warning, my package broke on a bug fix release of Python because of changes in Python. One solution *is* for the maintainers of affected packages to change their packaging and documentation to explicitly require Distribute. That's not going to happen overnight, of course. So is it possible to get a better idea of what the impact of this problem is going to be? I've already identified one real-world use case: (1) OS X users who install 2.6.3 from python.org and who need to install certain packages (lxml, appscript, zope-interfaces, others?). On what other platforms is this likely to be a problem? Windows *? Linuxes? If that can be identified, if necessary the distributors of Python installers can be informed so they can inform their users (note, that python.org is itself a distributor of Python installers). I don't have the expertise to make an assessment of that. Is it possible to get a better idea of what packages might be affected, i.e. what is it that these apps are using in their setup.py, or elsewhere, that causes the problem to appear? If so, it would be easier to warn the affected developers and users. Certainly some kind of warning could be added to the python.org 2.6.3 download pages and sent out of the various news lists. Perhaps a note about 2.6.3 and 3.x could be added to the original setuptools / easy_install PyPi page. I guess the bottom line question is: Is there anything more pro-active that should be done? -- Ned Deily, n...@acm.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
On Sat, 03 Oct 2009 14:17:50 -0700, Ned Deily n...@acm.org wrote: On what other platforms is this likely to be a problem? Windows *? Linuxes? If that can be identified, if necessary the distributors of Python installers can be informed so they can inform their users (note, that python.org is itself a distributor of Python installers). I don't have the expertise to make an assessment of that. From what I've seen, it affects all platforms. Is it possible to get a better idea of what packages might be affected, i.e. what is it that these apps are using in their setup.py, or elsewhere, that causes the problem to appear? I believe whenever the build_ext command is used, this problem would occur. One trivial package that fails is zope.interface. -srid ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Package install failures in 2.6.3 - setuptools vs Distribute
2009/10/3 Ned Deily n...@acm.org: That's fine but they're not going to know about Distribute unless they stumble across discussions like this. They are going to ask around, and somebody will know. Most reasonably, they are going to ask the maker of the module they are trying to install, and say Hey your module doesn't work with Python 2.6.3. And that module guy will ask the python list. And the python list will know. People are not isolated. Information will reach them. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
