Re: Python 3.4 and ensurepip (rehashed, long)
On Mar 21, 2014, at 04:30 AM, Bohuslav Kabrda wrote: I'm on python-dev but not pypa-dev. But if you want to take the discussion there, I guess I'm ok with following yet another mailing list. Gmane makes life on a zillion mailing lists so much more manageable. ;) Yep, they do that. That's actually another thing that we're working on right now - creating an upstream-acceptable patch that would make all system-wide pip installations go into /usr/local/lib/..., while system-wide RPM installed packages would install into /usr/lib/... We know that Debian already has some downstream patches for this and there is also a bug opened for this upstream [1]. But yeah, currently users just sudo pip install foo into system Python packages path. Many Pycons ago, Toshio, myself, Tarek, and several others discussed a scheme where by (through more indirection of course), various packaging tools could query configuration files to determine operating system specific paths. I thought it was quite a nice scheme which would have let us easily drop a conf file to declare where Debian wanted certain files installed, and Fedora or OS X, etc could declare others. Mostly this was in the context of the (now abandoned) successor to setup.py, but I think tools like pip and such could have taken advantage of that. I'd still like something like that. On Debian, Python adds a /usr/local directory to sys.path, but it's not /usr/local/lib/pythonX.Y/site-packages, it's .../dist-packages for reasons we've long hashed out and I don't want to re-open. Suffice to say that we do have a directory where `sudo pip install` could install to that would play nice with system Python but wouldn't interfere with distro packages. While I don't much like `sudo pip install` as a general recommendation, in principle we support that. Just to keep in mind when proposing this upstream that distros will have similar goals, but different concrete paths. What will rewheeling prefer when: * bundled ones are newer than system ones? That's not currently implemented in our patch, since we'll be removing bundled wheels and relying on system versions only. But to solve this situation for the upstream patch, we'll do it like this - if user runs just python3 -m ensurepip, nothing will happen - if user runs python3 -m ensurepip --upgrade, his setuptools/pip will get upgraded Note, that this corresponds with current ensurepip documentation [2] of such situations. I read the patch, and I think I get it, but just to be clear, rewheeling only covers pip and setuptools, not arbitrary other packages, right? I still don't like introducing the circular dependency in Debian, so Piotr's suggestion is about where I think Debian should head. That is: `python3 -m ensurepip` would suggest (but not execute) to install python3-pip (our binary package providing pip for Python 3) if it's not already installed. The key thing is to make sure that pyvenv works in both cases. I'm not exactly sure about --upgrade. I'm not sure what we would do if we wanted avoided the dependency cycle, and pip/setuptools wasn't yet installed system wide. Yeah, that's a tough one and it's actually one of the reasons why we decided to introduce the dependency cycle. The obvious (and IMO wrong) solution would be subprocess.Popen(['sudo', 'yum', 'install', 'python-pip']), but that just isn't something that I'd want to do. Right, we wouldn't either I think. Cheers, -Barry signature.asc Description: PGP signature
Re: Python 3.4 and ensurepip (rehashed, long)
On Mar 21, 2014, at 02:43 PM, Donald Stufft wrote: Also yea pip in the system Python currently kind of sucks. I want to make this better eventually! I just don't know how yet or have the cycles to spend investigating it. One of the things I'd like to see, in addition to supporting platform-specified system-level installation directories (i.e. /usr/local/lib/pythonX.Y/dist-packages on Debian), is an upstream switch to tell Python to ignore this directory, mirroring e.g. -s. That way, if people say Well I just pip installed foo into /usr/local and it broke my system we'd be able to respond you better use pythonX.Y --dont-blame-us. This is especially important for system services written in Python, and it's analogous to how we recommend using -Es on the shebang line for many of these services. Or put another way: it's okay if users need extra stuff that the OS doesn't package, and it's okay if they want to install them in such a way that all users will benefit from them, but the OS must have a way to isolate its environment from these user choices. Cheers, -Barry signature.asc Description: PGP signature
Re: Python 3.4 and ensurepip (rehashed, long)
On Mar 21, 2014, at 02:28 PM, Matthias Klose wrote: - whether to use python3-pip or a freshly downloaded pip. When using pip to install for the system python3, maybe try to use the shipped python3-pip. I have no opinion if another pip is downloaded and installed into for environments created by venv or virtualenv. Some package builds use virtual environments, but the population of these has to be satisfied by package build dependencies anyway. In general, I'd like to use the Debian package pip/setuptools for installation into system locations, and I am okay with using bundled pip/setuptools for installation into virtualenvs. I think that's pretty much the current state of affairs anyway, except for pyvenv. Cheers, -Barry -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140325151821.438fa...@limelight.wooz.org
Re: Python 3.4 and ensurepip (rehashed, long)
On Mar 25, 2014, at 3:11 PM, Barry Warsaw ba...@debian.org wrote: On Mar 21, 2014, at 04:30 AM, Bohuslav Kabrda wrote: I'm on python-dev but not pypa-dev. But if you want to take the discussion there, I guess I'm ok with following yet another mailing list. Gmane makes life on a zillion mailing lists so much more manageable. ;) Yep, they do that. That's actually another thing that we're working on right now - creating an upstream-acceptable patch that would make all system-wide pip installations go into /usr/local/lib/..., while system-wide RPM installed packages would install into /usr/lib/... We know that Debian already has some downstream patches for this and there is also a bug opened for this upstream [1]. But yeah, currently users just sudo pip install foo into system Python packages path. Many Pycons ago, Toshio, myself, Tarek, and several others discussed a scheme where by (through more indirection of course), various packaging tools could query configuration files to determine operating system specific paths. I thought it was quite a nice scheme which would have let us easily drop a conf file to declare where Debian wanted certain files installed, and Fedora or OS X, etc could declare others. Mostly this was in the context of the (now abandoned) successor to setup.py, but I think tools like pip and such could have taken advantage of that. I'd still like something like that. On Debian, Python adds a /usr/local directory to sys.path, but it's not /usr/local/lib/pythonX.Y/site-packages, it's .../dist-packages for reasons we've long hashed out and I don't want to re-open. Suffice to say that we do have a directory where `sudo pip install` could install to that would play nice with system Python but wouldn't interfere with distro packages. While I don't much like `sudo pip install` as a general recommendation, in principle we support that. Just to keep in mind when proposing this upstream that distros will have similar goals, but different concrete paths. I have a half formed idea in my head about a PEP that adds more directories on sys.path and the relevant APIs to query them (pip doesn’t decide where to install stuff, it just asks distutils where to install stuff). What will rewheeling prefer when: * bundled ones are newer than system ones? That's not currently implemented in our patch, since we'll be removing bundled wheels and relying on system versions only. But to solve this situation for the upstream patch, we'll do it like this - if user runs just python3 -m ensurepip, nothing will happen - if user runs python3 -m ensurepip --upgrade, his setuptools/pip will get upgraded Note, that this corresponds with current ensurepip documentation [2] of such situations. I read the patch, and I think I get it, but just to be clear, rewheeling only covers pip and setuptools, not arbitrary other packages, right? As far as I understand it yes, also Fedora’s python-pip still bundles pip._vendor.* I still don't like introducing the circular dependency in Debian, so Piotr's suggestion is about where I think Debian should head. That is: `python3 -m ensurepip` would suggest (but not execute) to install python3-pip (our binary package providing pip for Python 3) if it's not already installed. The key thing is to make sure that pyvenv works in both cases. I'm not exactly sure about —upgrade. I assume once someone has installed pip with apt-get they’d still be able to run pip install —upgrade pip if they wanted too? I'm not sure what we would do if we wanted avoided the dependency cycle, and pip/setuptools wasn't yet installed system wide. Yeah, that's a tough one and it's actually one of the reasons why we decided to introduce the dependency cycle. The obvious (and IMO wrong) solution would be subprocess.Popen(['sudo', 'yum', 'install', 'python-pip']), but that just isn't something that I'd want to do. Right, we wouldn't either I think. Cheers, -Barry - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Python 3.4 and ensurepip (rehashed, long)
On Mar 25, 2014, at 3:18 PM, Barry Warsaw ba...@debian.org wrote: On Mar 21, 2014, at 02:28 PM, Matthias Klose wrote: - whether to use python3-pip or a freshly downloaded pip. When using pip to install for the system python3, maybe try to use the shipped python3-pip. I have no opinion if another pip is downloaded and installed into for environments created by venv or virtualenv. Some package builds use virtual environments, but the population of these has to be satisfied by package build dependencies anyway. In general, I'd like to use the Debian package pip/setuptools for installation into system locations, and I am okay with using bundled pip/setuptools for installation into virtualenvs. I think that's pretty much the current state of affairs anyway, except for pyvenv. Doing what you do for system locations and bundling a Python package is the status quo for virtualenv, which the ensurepip/venv combo copied that technique. Cheers, -Barry -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140325151821.438fa...@limelight.wooz.org - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Python 3.4 and ensurepip (rehashed, long)
On Mar 25, 2014, at 03:19 PM, Donald Stufft wrote: I assume once someone has installed pip with apt-get they’d still be able to run pip install —upgrade pip if they wanted too? I would think they should be able to do that. -Barry signature.asc Description: PGP signature
Re: Python 3.4 and ensurepip (rehashed, long)
On Tuesday, March 25, 2014 15:29:06 Barry Warsaw wrote: On Mar 25, 2014, at 03:19 PM, Donald Stufft wrote: I assume once someone has installed pip with apt-get they’d still be able to run pip install —upgrade pip if they wanted too? I would think they should be able to do that. If I've install a package and it's upgraded (this is for the system, not for any kind of virtualized/isolated environment), I would find it quite surprising and unfortunate that it upgraded itself from an external source. Scott K signature.asc Description: This is a digitally signed message part.
Re: Python 3.4 and ensurepip (rehashed, long)
On 26 March 2014 17:15, Scott Kitterman deb...@kitterman.com wrote: On Tuesday, March 25, 2014 15:29:06 Barry Warsaw wrote: On Mar 25, 2014, at 03:19 PM, Donald Stufft wrote: I assume once someone has installed pip with apt-get they'd still be able to run pip install --upgrade pip if they wanted too? I would think they should be able to do that. If I've install a package and it's upgraded (this is for the system, not for any kind of virtualized/isolated environment), I would find it quite surprising and unfortunate that it upgraded itself from an external source. I'd be very surprised if a package manager told to upgrade itself used a different source for its own code vs things it manages. Yes, people that use pip to install things globally deserve to keep both pieces, but either prohibit it entirely, or have it work as advertised, not some frankenstein. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/caj3hoz28tp5udk7inf76rmjxqxc7xy_7ca0ls4p_meqs3m-...@mail.gmail.com
