Re: [Distutils] distil 0.1.1 released
On 3 May 2013 11:39, Donald Stufft don...@stufft.io wrote: On May 2, 2013, at 9:02 PM, Nick Coghlan ncogh...@gmail.com wrote: On Fri, May 3, 2013 at 9:30 AM, Donald Stufft don...@stufft.io wrote: On May 2, 2013, at 7:17 PM, Nick Coghlan ncogh...@gmail.com wrote: On 3 May 2013 09:08, Donald Stufft don...@stufft.io wrote: On May 2, 2013, at 6:48 PM, Nick Coghlan ncogh...@gmail.com wrote: pip --user manipulates the per-user directories (and I believe I filed a ticket a while back suggesting it should migrate to the model distil now uses). Cheers, Nick. I don't think it makes sense for pip to do this tbh. Language level installers should leave /usr alone by default, as that part of the filesystem is the domain of system packages. Cheers, Nick. Out of the three primary OSs only one of them has a concept of system packages. Regardless it's a major change in behavior and isn't a clear cut right or wrong choice. It's going to need a stronger justification than that to break backwards compatibility in such a large way. How about: I won't approve PEP 439 as long as pip still writes to the system site-packages by default when running as an ordinary user on Linux systems? sudo pip needs to be banished from the command line lexicon of Python developers. The transition isn't that hard: add --system now, switch the default from --system to --user for anyone *other than root* some time before Python 3.4. Anyone using virtual environments or an explicit --user or --system will be utterly unaffected when the default changes, as will anyone running an explicit sudo pip (in order to get write access to the system Python site-packages). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia So your answer to my wanting stronger justification is because I said so? Because sysadmins don't participate in upstream development and non-platform provided tools messing with /usr is wrong, wrong, wrong. The system Python is the domain of distro tools, we should be leaving it the hell alone. Cheers, Nick. *Why* does ``sudo pip`` need to be banished? ``pip install --user`` is not any safer, it's not any more or less wrong. It's an option. If pip was brand new I'd probably be of the opinion that I prefer the user scheme by default. However pip is not new, and it has a long history of working in this way. Changes can be made but I'm loath to support a large change in behavior like this without strong justification as to *why*. I also think your proposed solution of --user being the default unless we are running as root is suboptimal. Conditional defaults are confusing to end users, even more so when a common error condition (e.g. I forgot to type sudo before I pip installed, or I forgot to activate my virtualenv) is going to move from being an error to installing to the user scheme and *working*… right up until the time you actually try and use it. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] distil 0.1.1 released
On May 3, 2013, at 6:29 AM, Nick Coghlan ncogh...@gmail.com wrote: Because sysadmins don't participate in upstream development and non-platform provided tools messing with /usr is wrong, wrong, wrong. The system Python is the domain of distro tools, we should be leaving it the hell alone. Cheers, Nick. To be honest your proposal doesn't really do that since you want it to install into the site-packages if they are running it as root. If that's you're argument you should at least be consistent within your own proposal. One of my main problems with your proposal is that we suddenly have surprising behavior about where packages are being installed to: Linux + root user = site-packages Linux + regular user = user-packages (-- Going to confuse people because it's different) Linux + regular user + virtualenv = site-packages I don't know much about RHEL, but Debian installing globally means installing into /usr/local/lib/pythonX.Y/site-packages/ which is exactly where Debian wants you to put python libraries not installed via apt. Apt drops things into /usr/local/lib/pythonX.Y/dist-packages. Even if installing into the system python globally is wrong is there even a way too determine if we are attempting to install into a *system* python or not? I could be wrong but afaik there's not a good way to determine if this is a python that is managed by the system or one that was installed otherwise. If that's the case then we break by default one of the viable methods of isolation *from* system. This is very popular on OSX where people tend to install a non system python via homebrew to isolate themselves. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] distil 0.1.1 released
How about: I won't approve PEP 439 as long as pip still writes to the system site-packages by default when running as an ordinary user on Linux systems? sudo pip needs to be banished from the command line lexicon of Python developers. What is the proposal for Windows here? Would the default behaviour change? There's no concept of sudo on Windows. Personally, I don't care much as I only use pip in virtualenvs, but backward compatibility *is* important, even if the current behaviour is considered wrong. Also, don't forget that the current release of pip supports Python 2.5, which doesn't have per-user site-packages. The next release will drop support for 2.5, so we *could* start the switch to the user directory at the same time, but up until now that wasn't an option. PEP 439 may be about Python 3.4, but Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] distil 0.1.1 released
I would also be relatively happy for pip to refuse the temptation to guess if run globally and require an explicit --user or --system whenever it is run outside a virtual environment. However, I think it's better to make the typical pip install whatever work for most unprivileged users without requiring elevated privileges. I agree the proposed exception for root doesn't make sense so I withdraw that idea, even though installing things into root's home directory is a little strange. As far as Debian's dist-packages setup goes, that's their workaround for this misfeature of the current Python packaging ecosystem. Cheers, Nick. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] distil 0.1.1 released
On May 3, 2013, at 2:14 PM, Nick Coghlan ncogh...@gmail.com wrote: I would also be relatively happy for pip to refuse the temptation to guess if run globally and require an explicit --user or --system whenever it is run outside a virtual environment. However, I think it's better to make the typical pip install whatever work for most unprivileged users without requiring elevated privileges. I agree the proposed exception for root doesn't make sense so I withdraw that idea, even though installing things into root's home directory is a little strange. As far as Debian's dist-packages setup goes, that's their workaround for this misfeature of the current Python packaging ecosystem. Cheers, Nick. How do you determine run globally? - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] distil 0.1.1 released
On May 3, 2013, at 11:14 AM, Nick Coghlan ncogh...@gmail.com wrote: I would also be relatively happy for pip to refuse the temptation to guess if run globally and require an explicit --user or --system whenever it is run outside a virtual environment. However, I think it's better to make the typical pip install whatever work for most unprivileged users without requiring elevated privileges. I agree the proposed exception for root doesn't make sense so I withdraw that idea, even though installing things into root's home directory is a little strange. As far as Debian's dist-packages setup goes, that's their workaround for this misfeature of the current Python packaging ecosystem. As someone responsible for working with Python app deployment tools, this _will_ break the universe. Yes people should be using virtualenv, however many don't, deal with it. People expect package install as root to work like every other package system (yum, apt, take your pick) where they run sudo pip install a b c d and then run their app as a service-specific user. Magically writing to ~root will clearly not work in this case unless you also run your app as root (though I know some people do that too, but not behavior that should be encouraged). This proposal is entirely non-viable for anything but 100% best-practices users, full stop. --Noah signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] distil 0.1.1 released
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/03/2013 02:24 PM, Noah Kantrowitz wrote: People expect package install as root to work like every other package system (yum, apt, take your pick) where they run sudo pip install a b c d and then run their app as a service-specific user. I don't know any of those people: either folks have learned *not* to install as root, or else they run as root too. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlGEAP4ACgkQ+gerLs4ltQ6odACfXcO7KHBYvPJhWOZCXMtMTZsn bxIAn3j9jwRe70de+O0mIY4K2kfDrfzs =WgtN -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] distil 0.1.1 released
On May 3, 2013, at 2:25 PM, Tres Seaver tsea...@palladion.com wrote: Signed PGP part On 05/03/2013 02:24 PM, Noah Kantrowitz wrote: People expect package install as root to work like every other package system (yum, apt, take your pick) where they run sudo pip install a b c d and then run their app as a service-specific user. I don't know any of those people: either folks have learned *not* to install as root, or else they run as root too. I know a fair number of people like that. The common wisdom is *not* to install python packages from the system package mangers. because they generally either don't have everything so you'll need to install from PyPI anyways, or they'll have really old versions. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] distil 0.1.1 released
Hi Nick On 05/03/2013 12:14 PM, Nick Coghlan wrote: I would also be relatively happy for pip to refuse the temptation to guess if run globally and require an explicit --user or --system whenever it is run outside a virtual environment. However, I think it's better to make the typical pip install whatever work for most unprivileged users without requiring elevated privileges. I agree the proposed exception for root doesn't make sense so I withdraw that idea, even though installing things into root's home directory is a little strange. As far as Debian's dist-packages setup goes, that's their workaround for this misfeature of the current Python packaging ecosystem. IMO requiring either --user or --system is bad UI (we should have a sane default). Defaulting to --user is tempting, but also a bad idea, both because of the massive adjustment that it would require from existing users, and because it does the wrong thing in many cases where there are multiple Pythons compiled on the system. I think the best solution to this problem is quite easily implemented and non-intrusive: an improved error message when pip finds that it lacks permission to install to site-packages. This error message could recommend --user and/or virtualenv, could warn quite strongly against installing to this location unless you really know what you are doing, etc. Also, I think that for the specific situation of a Linux distribution's system Python (which, as Donald has pointed out, is only one mode of Python installation out of many), Debian's solution is better than just a workaround, it is conceptually more or less the Right Thing: separate distro-managed Python code from user-managed Python code, without preventing independent installation of Python-installation-wide code by the user. (Though some aspects of the implementation of Debian's solution are sub-optimal and have required ugly workarounds in pip. It would be better if multiple site-packages were a first-class concept in Python's own site.py so Debian's solution could be implemented via configuration rather than patches.) Carl ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] PEP 439 and default install locations (was Re: distil 0.1.1 released)
On 4 May 2013 09:03, Carl Meyer c...@oddbird.net wrote: Hi Nick On 05/03/2013 12:14 PM, Nick Coghlan wrote: I would also be relatively happy for pip to refuse the temptation to guess if run globally and require an explicit --user or --system whenever it is run outside a virtual environment. However, I think it's better to make the typical pip install whatever work for most unprivileged users without requiring elevated privileges. I agree the proposed exception for root doesn't make sense so I withdraw that idea, even though installing things into root's home directory is a little strange. As far as Debian's dist-packages setup goes, that's their workaround for this misfeature of the current Python packaging ecosystem. IMO requiring either --user or --system is bad UI (we should have a sane default). Defaulting to --user is tempting, but also a bad idea, both because of the massive adjustment that it would require from existing users, and because it does the wrong thing in many cases where there are multiple Pythons compiled on the system. I think the best solution to this problem is quite easily implemented and non-intrusive: an improved error message when pip finds that it lacks permission to install to site-packages. This error message could recommend --user and/or virtualenv, could warn quite strongly against installing to this location unless you really know what you are doing, etc. That sounds eminently reasonable. Also, I think that for the specific situation of a Linux distribution's system Python (which, as Donald has pointed out, is only one mode of Python installation out of many), Debian's solution is better than just a workaround, it is conceptually more or less the Right Thing: separate distro-managed Python code from user-managed Python code, without preventing independent installation of Python-installation-wide code by the user. (Though some aspects of the implementation of Debian's solution are sub-optimal and have required ugly workarounds in pip. It would be better if multiple site-packages were a first-class concept in Python's own site.py so Debian's solution could be implemented via configuration rather than patches.) Agreed, blessing something like Debian's solution sounds like a better idea than changing pip's default installation location. To provide that blessing, I think PEP 439 should be expanded to explicitly cover this aspect (there's another subtlety, in that it may be desirable for distro packages to be available when running with -S - I'm not sure how Debian currently handle that). Essentially, my goal is to have a clearly defined, consistent, cross-distro-and-platform shadowing behaviour for cases where you install a Python software distribution from a system package manager *and* through pip. The current situation, where the system tools and the Python specific tools tread on each other's toes at the filesystem layer (unless you change the default Python path, as Debian does) isn't appropriate for something we are going to ship as part of the standard Python toolkit. (Coincidentally, this may lead to my CPython bootstrapping changes becoming packaging related!) Cheers, Nick. Carl ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
