Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On 13 Jul, 2013, at 7:31, Nick Coghlan ncogh...@gmail.com wrote: 3. That means there are two main options available to us that I still consider viable alternatives (the installer bundling idea was suggested in one of the off list comments I mentioned): * an explicit bootstrapping script * bundling a *full* copy of pip with the Python installers for Windows and Mac OS X, but installing it to site-packages rather than to the standard library directory. That way pip can be used to upgrade itself as normal, rather than making it part of the standard library per se. This is then closer to the bundled application model adopted for IDLE in PEP 434 (we could, in fact, move to distributing idle the same way). Or automaticly invoke the bootstrap script during installation (for the Python installers), that we the installers don't end up with a stale version of pip. Either option should be easy enough to add to the OSX installers. Ronald ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On 13 Jul, 2013, at 15:35, Brett Cannon br...@python.org wrote: On Sat, Jul 13, 2013 at 2:29 AM, Ned Deily n...@acm.org wrote: In article 55b209b3-9576-4cf0-b58c-2a1e692af...@stufft.io, Donald Stufft don...@stufft.io wrote: On Jul 13, 2013, at 1:31 AM, Nick Coghlan ncogh...@gmail.com wrote: I'm currently leaning towards offering both, as we're going to need a tool for bootstrapping source builds, but the simplest way to bootstrap pip for Windows and Mac OS X users is to just *bundle a copy with the binary installers*. So long as the bundled copy looks *exactly* the way it would if installed later (so it can update itself), then we avoid the problem of coupling the pip update cycles to the standard library feature release cycle. The bundled version can be updated to the latest available versions when we do a Python maintenance release. Off the top of my head, including a copy of pip as a pre-installed global site-package seems like a very reasonable suggestion. For the python.org OS X installer, it should be no problem to implement. It would be equally easy to implement for future 2.7 and 3.3 maintenance releases. Does Apple just install the python.org OS X installer for distribution, or do they build their own thing? They do their own thing. My only worry is that Apple will not get the message about including pip and we will end up with an odd skew on OS X (I'm not worried about Linux distros as they all seem to follow Python development closely). That will happen anyway, pip won't get magically installed on current OSX releases and adding it to the upcoming 10.9 release is probably not possible either unless it already happens to be in the current beta (they appear to have a fairly early cutoff point for including new software at the Unix layer). Ronald ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On 13 Jul, 2013, at 16:39, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: I smoke-tested the script on vanilla Python 3.3 installations on Windows and OS X. On OS X, the pip script was written to the appropriate Frameworks folder, but not to /usr/local/bin - I assume it would be simple enough to arrange that? Not installing in /usr/local/bin is a feature that makes it easier to maintain several python installs without worrying about contamination (for example Python 3 and Python 2, or even 2.6 and 2.7). The installer by default adds the framework 'bin' directory to the environment for the shell of the user that installed the framework. Ronald ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On 15 July 2013 19:30, Ronald Oussoren ronaldousso...@mac.com wrote: On 13 Jul, 2013, at 7:31, Nick Coghlan ncogh...@gmail.com wrote: 3. That means there are two main options available to us that I still consider viable alternatives (the installer bundling idea was suggested in one of the off list comments I mentioned): * an explicit bootstrapping script * bundling a *full* copy of pip with the Python installers for Windows and Mac OS X, but installing it to site-packages rather than to the standard library directory. That way pip can be used to upgrade itself as normal, rather than making it part of the standard library per se. This is then closer to the bundled application model adopted for IDLE in PEP 434 (we could, in fact, move to distributing idle the same way). Or automaticly invoke the bootstrap script during installation (for the Python installers), that we the installers don't end up with a stale version of pip. Yeah, I see pros and cons to either approach, with the main con of install time bootstrapping being the requirement for network access, while the main con of bundling is that you may end up needing to do pip install --upgrade pip immediately after installing Python anyway. I currently have a slight preference for actual bundling, but could probably be persuaded to endorse an install time bootstrap instead. It's only the bootstrap-on-first-use approach that I've decided is asking for trouble. I don't believe either Martin (von Löwis) or Brian (Curtin) is on this list, so I'll email them directly to see if they have a preference. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
Not installing in /usr/local/bin is a feature that makes it easier to maintain several python installs without worrying about contamination (for example Python 3 and Python 2, or even 2.6 and 2.7). I thought it might be behaving as designed, but wasn't sure. Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On Jul 13, 2013, at 1:31 AM, Nick Coghlan ncogh...@gmail.com wrote: I'm currently leaning towards offering both, as we're going to need a tool for bootstrapping source builds, but the simplest way to bootstrap pip for Windows and Mac OS X users is to just *bundle a copy with the binary installers*. So long as the bundled copy looks *exactly* the way it would if installed later (so it can update itself), then we avoid the problem of coupling the pip update cycles to the standard library feature release cycle. The bundled version can be updated to the latest available versions when we do a Python maintenance release. We could simply check it into the site-packages inside the CPython source tree could we not? *Not* providing a bootstrap script and merely checking it into the default site-packages means it's available for everyone. No matter how python installed. If Linux packagers really don't want it installed by default they could simply just remove it and either install it along with Python, or continue to keep it how it is today as a separate package? There are a number of things that have to be taken into account when downloading pip from the internet that are completely side stepped when we well don't download it from the internet :) And bundling it as a pre-installed python module and not in the standard library solves basically all of the problems I have with putting it inside of Python. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On Jul 13, 2013, at 1:57 AM, Noah Kantrowitz n...@coderanger.net wrote: I would absolutely advocate to packagers that installing the main python package results in a working pip install, regardless of how that is accomplished. +1 to this as well. Ideally, if we go down this route, installing python just comes with pip preinstalled. However that takes place :) - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
In article 55b209b3-9576-4cf0-b58c-2a1e692af...@stufft.io, Donald Stufft don...@stufft.io wrote: On Jul 13, 2013, at 1:31 AM, Nick Coghlan ncogh...@gmail.com wrote: I'm currently leaning towards offering both, as we're going to need a tool for bootstrapping source builds, but the simplest way to bootstrap pip for Windows and Mac OS X users is to just *bundle a copy with the binary installers*. So long as the bundled copy looks *exactly* the way it would if installed later (so it can update itself), then we avoid the problem of coupling the pip update cycles to the standard library feature release cycle. The bundled version can be updated to the latest available versions when we do a Python maintenance release. Off the top of my head, including a copy of pip as a pre-installed global site-package seems like a very reasonable suggestion. For the python.org OS X installer, it should be no problem to implement. It would be equally easy to implement for future 2.7 and 3.3 maintenance releases. We could simply check it into the site-packages inside the CPython source tree could we not? *Not* providing a bootstrap script and merely checking it into the default site-packages means it's available for everyone. No matter how python installed. If Linux packagers really don't want it installed by default they could simply just remove it and either install it along with Python, or continue to keep it how it is today as a separate package? This sounds an unnecessary complication. I suspect that there is a small minority of users who actually build Python from source. And they should know what they are doing. I believe most users either use a distribution-provided Python (via their OS) or a third-party package provider (including python.org binary installers and their derivatives). The OS distributors are going to do what they currently do; the only change needed is to persuade them to include their pip package as a mandatory dependency. Trying to hack the Python source build process to include a copy of pip is just not worth the effort. -- Ned Deily, n...@acm.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On 13 July 2013 16:05, Donald Stufft don...@stufft.io wrote: On Jul 13, 2013, at 1:31 AM, Nick Coghlan ncogh...@gmail.com wrote: I'm currently leaning towards offering both, as we're going to need a tool for bootstrapping source builds, but the simplest way to bootstrap pip for Windows and Mac OS X users is to just *bundle a copy with the binary installers*. So long as the bundled copy looks *exactly* the way it would if installed later (so it can update itself), then we avoid the problem of coupling the pip update cycles to the standard library feature release cycle. The bundled version can be updated to the latest available versions when we do a Python maintenance release. We could simply check it into the site-packages inside the CPython source tree could we not? *Not* providing a bootstrap script and merely checking it into the default site-packages means it's available for everyone. No matter how python installed. Source code that isn't maintained through bugs.python.org isn't getting checked into the CPython repo - see PEP 360. Getting the latest released version of something from PyPI is a different story, though. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On Sat, Jul 13, 2013 at 8:07 AM, Donald Stufft don...@stufft.io wrote: +1 to this as well. Ideally, if we go down this route, installing python just comes with pip preinstalled. However that takes place :) Well, I don't want a however that takes place that causes more packaging problems in the future, so I'm -1 on that. :-) I think including it in binary installers in such a way that pip can upgrade itself is definitely a good idea. Including it in the source code distributions seems less of a benefit. In fact, I don't even think I want it as things stand right now. Story time: This week a lot of people had problems with zc.buildout, because of the new releases of setuptools and distribute. Essentially they tried to upgrade themselves and failed for various reasons, such as distribute or setuptools being installed in the system python and not being upgradeable, or not being upgradeable for other reasons, like permissions. Mostly, I had no problems, because the Python installs I use have only one package installed: Virtualenv. That means that for most buildouts I used, buildout had istelf installed setuptools or distribute in its own isolated way. I only got errors where an overly complex build tool insisted on making virtualenvs, installing distribute in them and then running buildout with those virtualenvs. In those cases I had to run bin/pip -U distribute, so it was also an easy fix. So by default I prefer to have no packages except virtualenv for my source-built Pythons. This isn't a big issue, and I'm not gonna throw a hissyfit or anything, but I think I'll -1 the idea of including pip in a source install. Having pyvenv install pip by default in the new environment is +lots from me though. Also, having pyvenv be able to install itself in a non-empty directory would be great. ;-) //Lennart ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On 13 July 2013 06:31, Nick Coghlan ncogh...@gmail.com wrote: * bundling a *full* copy of pip with the Python installers for Windows and Mac OS X, but installing it to site-packages rather than to the standard library directory. That way pip can be used to upgrade itself as normal, rather than making it part of the standard library per se. This is then closer to the bundled application model adopted for IDLE in PEP 434 (we could, in fact, move to distributing idle the same way). How robust is the process of upgrading pip using itself? Specifically on Windows, where these things typically seem less reliable. Personally, I have never upgraded pip using itself, because I only ever install pip in virtualenvs, which don't have a lifespan as long as a pip release cycle :-) It would be easy to imagine a new pip release resulting in a *lot* of bugs raised against Python (rather than pip) saying that the upgrade fails. And of course if an upgrade fails, we can't just release a new version of pip that fixes the issue, because it's the *old* version that is installed and has to do the upgrade. So there's manual fiddling to do. Not a good experience for Python users. My current workflow is to have absolutely nothing installed in the system Python and use virtualenvs for everything. This is a bit extreme, but the issues I've hit in the past when package management has gone wrong have made me very cautious. If the pip upgrade process is rock-solid, this isn't an issue, but I'm not sure that it is, myself. Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On 13 July 2013 13:25, Nick Coghlan ncogh...@gmail.com wrote: I think we need to flip the dependencies so that pip as the installer has all the essential code for installation from PyPI and then setuptools and distlib depend on that pip infrastructure. No need to add anything to the standard library prematurely when we can add it to pip instead. Agreed, up to a point. What I've worked on in the past is things like automated wheel/sdist installers for systems with no internet access at all (distributions copied onto the PC via portable disk) or behind broken proxies (Internet Explorer works to download files from the net, but nothing else does). For those environments, the key to me is that I *only* use stuff that is available in a stock Python build. Something like pyton -m getpip *will not work*, so I have to roll my own means of bootstrapping. Of course I could copy a pip sdist to the machine, unpack it and run python setup.py install. More likely, I write bootstrap code in my script to do that automatically. It's a very specialised use case, certainly, and there are plenty of ways around the issues, but I have seen give up and just use VBScript./perl used as the fallback, as well, and that makes me sad... Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On Sat, Jul 13, 2013 at 1:31 AM, Nick Coghlan ncogh...@gmail.com wrote: In addition to the long thread based on Richard's latest set of updates, I've also received a few off-list comments on the current state of the proposal. So, I figured I'd start a new thread summarising my current point of view and see where we want to go from there. 1. However we end up solving the bootstrapping problem, I'm *definitely* a fan of us updating pyvenv in 3.4 to ensure that pip is available by default in new virtual environments created with that tool. I also have an idea for a related import system feature that I'll be sending to import-sig this afternoon (it's a variant on *.pth and *.egg-link files that should be able to address a variety of existing problems, including the one of *selectively* making system and user packages available in a virtual environment in a cross-platform way without needing to copy them) 2. While I was originally a fan of the implicit bootstrapping on demand design, I no longer like that notion. While Richard's bootstrap script is a very nice piece of work, the edge cases and neat tricks have built up to the point where they trip my if the implementation is hard to explain, it's a bad idea filter. Accordingly, I no longer think the implicit bootstrapping is a viable option. 3. That means there are two main options available to us that I still consider viable alternatives (the installer bundling idea was suggested in one of the off list comments I mentioned): * an explicit bootstrapping script * bundling a *full* copy of pip with the Python installers for Windows and Mac OS X, but installing it to site-packages rather than to the standard library directory. That way pip can be used to upgrade itself as normal, rather than making it part of the standard library per se. This is then closer to the bundled application model adopted for IDLE in PEP 434 (we could, in fact, move to distributing idle the same way). I'm currently leaning towards offering both, as we're going to need a tool for bootstrapping source builds, but the simplest way to bootstrap pip for Windows and Mac OS X users is to just *bundle a copy with the binary installers*. So long as the bundled copy looks *exactly* the way it would if installed later (so it can update itself), then we avoid the problem of coupling the pip update cycles to the standard library feature release cycle. The bundled version can be updated to the latest available versions when we do a Python maintenance release. For Linux, if you're using the system Python on a Debian or Fedora derivative, then sudo apt-get python-pip and sudo yum install python-pip are both straightforward, and if you're using something else, then it's unlikely getting pip bootstrapped using the bootstrap script is a task that will bother you :) The python -m getpip command is still something we will want to provide, as it is useful to people that build their own copy of Python from source. But is it going to make a difference? If we shift to using included copies of pip in binary installers over a bootstrap I say leave out the bootstrap as anyone building from source should know how to get pip installed on their machine or venv. The only reason I see it worth considering is if pyvenv starts bootstrapping pip and we want to support the case of pip not being installed. But if we are including it in the binary installer and are going to assume it's available through OS distros, then there isn't a need to as pip can then install pip for us into the venv and skip any initial pip bootstrap. If pip isn't found we can simply either point to the docs in the failure message or print out the one-liner it takes to install pip (and obviously there can be a --no-pip flag to skip this for people who want to install it manually like me who build from source). IOW I think taking the worldview in Python 3.4 that pip will come installed with Python unless you build from source negates the need for the bootstrap script beyond just saying ``curl https://pypi.python.org/get-pip.py | python`` if pip isn't found. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On Sat, Jul 13, 2013 at 9:38 AM, Paul Moore p.f.mo...@gmail.com wrote: On 13 July 2013 14:31, Brett Cannon br...@python.org wrote: +1 on the inversion. I don't know what that will do to pip, it makes sense to have the installer self-contained and the packaging/building libraries be something that you grab using the installer. Having to grab the packaging infrastructure to get an installer is the more painful route. TBH, I don't understand what the inversion implies. If it means pip taking all of the distlib/setuptools code that it currently uses, and making it part of pip and maintained within pip (essentially as a fork while the inversion is going on) then I'm not keen on that. Personally, I don't want to have to maintain that code myself - I guess if Vinay and Jason were pip maintainers and looked after that code, then that's an option. If it means pip vendoring distlib and setuptools, then OK (we do that for distlib already) The point is you shouldn't have to grab a packaging tool just to install stuff if you never need the packaging tool. Since pip is supposed to be *the* first thing you install for Python you don't want that to have its own dependencies, muddying up the installation process. but I don't see the benefit - no-ione should be doing from pip.vendor.distlib.version import Version. That's just asking for trouble if someone did that (plus if you did that it would be pip._vendor to get the privacy point across). I'd need to know better what it means for pip, I guess... I suspect we all do. =) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
Nick Coghlan ncoghlan at gmail.com writes: 1. However we end up solving the bootstrapping problem, I'm *definitely* a fan of us updating pyvenv in 3.4 to ensure that pip is available by default in new virtual environments created with that tool. Will that need green-lighting on python-dev? As events have shown, that script has needed updating between Python releases. OTOH, I'm not sure anyone could have predicted at 3.3 release time that setuptools and Distribute would kiss and make up :-) We should probably ensure that the pip and setuptools URLs used in that script (pyvenvex.py, to be renamed to pyvenv.py) are formally agreed with the relevant maintainers. Or, since those URLs just fetch the latest releases, perhaps some different URLs should be used which refer to more stable releases (for some definition of more stable) - perhaps those should be python.org URLs, rather than BitBucket and GitHub as they are at present. That way, changing the resources which those URLs reference would have to be an active decision by someone, rather than just following the latest developments on setuptools and pip. There are pluses and minuses either way, of course. 2. While I was originally a fan of the implicit bootstrapping on demand design, I no longer like that notion. While Richard's bootstrap script is a very nice piece of work, the edge cases and neat tricks have built up to the point where they trip my if the implementation is hard to explain, it's a bad idea filter. Accordingly, I no longer think the implicit bootstrapping is a viable option. But if your reservation stems from one specific implementation of the idea, then might not an alternative implementation fit the bill? Consider: the pyvenvex.script merely runs bootstrapping scripts from setuptools and pip in the venv - there's no magic. I couldn't see Richard's script referenced in the PEP, which just referred to the PIP issue tracker which had no obvious link to any commit or script. So I don't know what the edge cases and neat tricks are that you're referring to. I adapted the pyvenvex.py script into a getpip.py script, available here: https://gist.github.com/vsajip/5990837 82 lines all told - what cases does it not cover? It installs setuptools and pip into the system site-packages for the invoking Python, if not already present. It can, of course, be refined to e.g. install even if the packages are already present, which is tantamount to upgrading. I smoke-tested the script on vanilla Python 3.3 installations on Windows and OS X. On OS X, the pip script was written to the appropriate Frameworks folder, but not to /usr/local/bin - I assume it would be simple enough to arrange that? On Windows, the pip script (including Windows native launcher) were written to c:\Python33\Scripts. The bundling idea will obviously need to be discussed with the installer builders, and on python-dev in general If python-dev agrees to the updated pyvenv.py script, then this type of addition should be uncontentious, as it basically does the same thing. It seems a whole lot less work than bundling, to me. Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On 13 July 2013 13:25, Nick Coghlan ncogh...@gmail.com wrote: I think we need to flip the dependencies so that pip as the installer has all the essential code for installation from PyPI and then setuptools and distlib depend on that pip infrastructure. No need to add anything to the standard library prematurely when we can add it to pip instead. If we do this, I think people will start to expect to be able to code scripts to the pip API. (We've had people ask this on the pip tracker already). If we don't want pip to end up like distutils (with people depending on all sorts of random bits of the internals, because there's no documented API) as a backward-compatibility nightmare, we need to consider how to handle this. Of course, saying explicitly only the python -m pip command line interface is stable and supported may well be enough. But didn't we just say that setuptools and distlib depend on the pip API? So either they have special privileges (presumably because they are under the umbrella of the PyPA) or we can't avoid documenting/supporting some API... I don't believe that pip is currently in a state to offer a solid documented internal API. Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/13/2013 08:25 AM, Nick Coghlan wrote: I think we need to flip the dependencies so that pip as the installer has all the essential code for installation from PyPI and then setuptools and distlib depend on that pip infrastructure. No need to add anything to the standard library prematurely when we can add it to pip instead. - -1. That would effectively mean inlining the bulk of setuptools' code into pip (which is just a UI / policy shim over it). You might as well just have your bootstrapper install both pip and setuptools and be done Unless distlib (or something like it) lands in the stdlib with enough features to support a setuptools-less pip, of course. At-which-point-the-State-will-wither-away'ly, Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlHhbLkACgkQ+gerLs4ltQ51fQCfZrmZN5mJKrtoGFTk0YqQrBHd F/IAnRp6XjoU4SpXZ4v3Uz6iOBrCZZZn =gSA5 -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On Jul 13, 2013, at 11:00 AM, Paul Moore p.f.mo...@gmail.com wrote: Of course, saying explicitly only the python -m pip command line interface is stable and supported may well be enough. But didn't we just say that setuptools and distlib depend on the pip API? So either they have special privileges (presumably because they are under the umbrella of the PyPA) or we can't avoid documenting/supporting some API... I don't think we need this any more than if pip was not pre-installed. So still nice to have for things like chef but not a requirement. setuptools and distlib won't depend on pip, pip will just bundle them (like it already does for distlib). The idea should be that pip itself has no dependencies because a package manager with dependencies is kind of strange and can easily lead to issues where the package manager breaks and is unifiable (e.g. setuptools breaks and pip can't be used to fix it). - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On Jul 13, 2013, at 11:05 AM, Tres Seaver tsea...@palladion.com wrote: Signed PGP part On 07/13/2013 08:25 AM, Nick Coghlan wrote: I think we need to flip the dependencies so that pip as the installer has all the essential code for installation from PyPI and then setuptools and distlib depend on that pip infrastructure. No need to add anything to the standard library prematurely when we can add it to pip instead. - -1. That would effectively mean inlining the bulk of setuptools' code into pip (which is just a UI / policy shim over it). You might as well just have your bootstrapper install both pip and setuptools and be done Unless distlib (or something like it) lands in the stdlib with enough features to support a setuptools-less pip, of course. At-which-point-the-State-will-wither-away'ly, Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig I was planning on doing this to pip anyways. pip should not have any dependencies setuptools or otherwise. So regardless of what happens with this PEP I want pip to be inlining setuptools and providing the code to make that transparent during install. This would mean that for people who are _just_ installing packages they don't need setuptools installed. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On Jul 13, 2013, at 6:46 AM, Brett Cannon wrote: On Sat, Jul 13, 2013 at 1:31 AM, Nick Coghlan ncogh...@gmail.com wrote: In addition to the long thread based on Richard's latest set of updates, I've also received a few off-list comments on the current state of the proposal. So, I figured I'd start a new thread summarising my current point of view and see where we want to go from there. 1. However we end up solving the bootstrapping problem, I'm *definitely* a fan of us updating pyvenv in 3.4 to ensure that pip is available by default in new virtual environments created with that tool. I also have an idea for a related import system feature that I'll be sending to import-sig this afternoon (it's a variant on *.pth and *.egg-link files that should be able to address a variety of existing problems, including the one of *selectively* making system and user packages available in a virtual environment in a cross-platform way without needing to copy them) 2. While I was originally a fan of the implicit bootstrapping on demand design, I no longer like that notion. While Richard's bootstrap script is a very nice piece of work, the edge cases and neat tricks have built up to the point where they trip my if the implementation is hard to explain, it's a bad idea filter. Accordingly, I no longer think the implicit bootstrapping is a viable option. 3. That means there are two main options available to us that I still consider viable alternatives (the installer bundling idea was suggested in one of the off list comments I mentioned): * an explicit bootstrapping script * bundling a *full* copy of pip with the Python installers for Windows and Mac OS X, but installing it to site-packages rather than to the standard library directory. That way pip can be used to upgrade itself as normal, rather than making it part of the standard library per se. This is then closer to the bundled application model adopted for IDLE in PEP 434 (we could, in fact, move to distributing idle the same way). I'm currently leaning towards offering both, as we're going to need a tool for bootstrapping source builds, but the simplest way to bootstrap pip for Windows and Mac OS X users is to just *bundle a copy with the binary installers*. So long as the bundled copy looks *exactly* the way it would if installed later (so it can update itself), then we avoid the problem of coupling the pip update cycles to the standard library feature release cycle. The bundled version can be updated to the latest available versions when we do a Python maintenance release. For Linux, if you're using the system Python on a Debian or Fedora derivative, then sudo apt-get python-pip and sudo yum install python-pip are both straightforward, and if you're using something else, then it's unlikely getting pip bootstrapped using the bootstrap script is a task that will bother you :) The python -m getpip command is still something we will want to provide, as it is useful to people that build their own copy of Python from source. But is it going to make a difference? If we shift to using included copies of pip in binary installers over a bootstrap I say leave out the bootstrap as anyone building from source should know how to get pip installed on their machine or venv. The only reason I see it worth considering is if pyvenv starts bootstrapping pip and we want to support the case of pip not being installed. But if we are including it in the binary installer and are going to assume it's available through OS distros, then there isn't a need to as pip can then install pip for us into the venv and skip any initial pip bootstrap. If pip isn't found we can simply either point to the docs in the failure message or print out the one-liner it takes to install pip (and obviously there can be a --no-pip flag to skip this for people who want to install it manually like me who build from source). IOW I think taking the worldview in Python 3.4 that pip will come installed with Python unless you build from source negates the need for the bootstrap script beyond just saying ``curl https://pypi.python.org/get-pip.py | python`` if pip isn't found. This is highly unhelpful for dealing with systems automation. For the foreseeable future, the bulk of Python 3.4 installations will either be source installs, or homegrown packages based on source installs. The bundled pip doesn't need to be included with, say, an hg clone that you then build and install, but it does have to come with an install from an official release source tarball. --Noah signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Current status of PEP 439 (pip boostrapping)
In addition to the long thread based on Richard's latest set of updates, I've also received a few off-list comments on the current state of the proposal. So, I figured I'd start a new thread summarising my current point of view and see where we want to go from there. 1. However we end up solving the bootstrapping problem, I'm *definitely* a fan of us updating pyvenv in 3.4 to ensure that pip is available by default in new virtual environments created with that tool. I also have an idea for a related import system feature that I'll be sending to import-sig this afternoon (it's a variant on *.pth and *.egg-link files that should be able to address a variety of existing problems, including the one of *selectively* making system and user packages available in a virtual environment in a cross-platform way without needing to copy them) 2. While I was originally a fan of the implicit bootstrapping on demand design, I no longer like that notion. While Richard's bootstrap script is a very nice piece of work, the edge cases and neat tricks have built up to the point where they trip my if the implementation is hard to explain, it's a bad idea filter. Accordingly, I no longer think the implicit bootstrapping is a viable option. 3. That means there are two main options available to us that I still consider viable alternatives (the installer bundling idea was suggested in one of the off list comments I mentioned): * an explicit bootstrapping script * bundling a *full* copy of pip with the Python installers for Windows and Mac OS X, but installing it to site-packages rather than to the standard library directory. That way pip can be used to upgrade itself as normal, rather than making it part of the standard library per se. This is then closer to the bundled application model adopted for IDLE in PEP 434 (we could, in fact, move to distributing idle the same way). I'm currently leaning towards offering both, as we're going to need a tool for bootstrapping source builds, but the simplest way to bootstrap pip for Windows and Mac OS X users is to just *bundle a copy with the binary installers*. So long as the bundled copy looks *exactly* the way it would if installed later (so it can update itself), then we avoid the problem of coupling the pip update cycles to the standard library feature release cycle. The bundled version can be updated to the latest available versions when we do a Python maintenance release. For Linux, if you're using the system Python on a Debian or Fedora derivative, then sudo apt-get python-pip and sudo yum install python-pip are both straightforward, and if you're using something else, then it's unlikely getting pip bootstrapped using the bootstrap script is a task that will bother you :) The python -m getpip command is still something we will want to provide, as it is useful to people that build their own copy of Python from source. The bundling idea will obviously need to be discussed with the installer builders, and on python-dev in general, but that was always going to be the case for this PEP anyway (since it *does* touch CPython directly, rather than just being related to the packaging ecosystem). It achieves the aim of allowing people to assume some version of pip will be present on Python 3.4+ installations (or readily available in the case of Linux), while avoiding the problem of coupling pip updates to major Python version updates. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Current status of PEP 439 (pip boostrapping)
On Jul 12, 2013, at 10:31 PM, Nick Coghlan wrote: In addition to the long thread based on Richard's latest set of updates, I've also received a few off-list comments on the current state of the proposal. So, I figured I'd start a new thread summarising my current point of view and see where we want to go from there. 1. However we end up solving the bootstrapping problem, I'm *definitely* a fan of us updating pyvenv in 3.4 to ensure that pip is available by default in new virtual environments created with that tool. I also have an idea for a related import system feature that I'll be sending to import-sig this afternoon (it's a variant on *.pth and *.egg-link files that should be able to address a variety of existing problems, including the one of *selectively* making system and user packages available in a virtual environment in a cross-platform way without needing to copy them) 2. While I was originally a fan of the implicit bootstrapping on demand design, I no longer like that notion. While Richard's bootstrap script is a very nice piece of work, the edge cases and neat tricks have built up to the point where they trip my if the implementation is hard to explain, it's a bad idea filter. Accordingly, I no longer think the implicit bootstrapping is a viable option. 3. That means there are two main options available to us that I still consider viable alternatives (the installer bundling idea was suggested in one of the off list comments I mentioned): * an explicit bootstrapping script * bundling a *full* copy of pip with the Python installers for Windows and Mac OS X, but installing it to site-packages rather than to the standard library directory. That way pip can be used to upgrade itself as normal, rather than making it part of the standard library per se. This is then closer to the bundled application model adopted for IDLE in PEP 434 (we could, in fact, move to distributing idle the same way). I'm currently leaning towards offering both, as we're going to need a tool for bootstrapping source builds, but the simplest way to bootstrap pip for Windows and Mac OS X users is to just *bundle a copy with the binary installers*. So long as the bundled copy looks *exactly* the way it would if installed later (so it can update itself), then we avoid the problem of coupling the pip update cycles to the standard library feature release cycle. The bundled version can be updated to the latest available versions when we do a Python maintenance release. For Linux, if you're using the system Python on a Debian or Fedora derivative, then sudo apt-get python-pip and sudo yum install python-pip are both straightforward, and if you're using something else, then it's unlikely getting pip bootstrapped using the bootstrap script is a task that will bother you :) The python -m getpip command is still something we will want to provide, as it is useful to people that build their own copy of Python from source. The bundling idea will obviously need to be discussed with the installer builders, and on python-dev in general, but that was always going to be the case for this PEP anyway (since it *does* touch CPython directly, rather than just being related to the packaging ecosystem). It achieves the aim of allowing people to assume some version of pip will be present on Python 3.4+ installations (or readily available in the case of Linux), while avoiding the problem of coupling pip updates to major Python version updates. As someone that has otherwise remained silent on this thread but was talking with people off-list I probably owe them a public +1 for bundling pip as a semi-new category of non-stdlib-but-included project. This would bring us in line with other tools like gem and npm which work out of the box and gives the user experience people want. Care would have to be paid to make sure the final pip binary ends up in the right filename, much in the same way as we do python - python2 - python 2.7 and such, but this is a solvable problem. How linux distros adapt to this is certainly another question, but I would absolutely advocate to packagers that installing the main python package results in a working pip install, regardless of how that is accomplished. As someone that has to write system management scripts to install and configure Python, being able to count on both pip and pyenv as standard tools in standard places is near-mind-blowingly awesome (give or take that it would be many years unt il I could reasonably assume 3.4 as the default python, but a man can dream). While the getpip module is interesting in a few use cases, it is vastly more valuable to me that we focus on the user experience of the majority of Python developers and deployments, and this is somewhere that Ruby and Node are getting it right in having the package tool simply be there by default. Bundling also addresses the myriad
