On Tue, Mar 19, 2013 at 3:37 PM, Daniel Holth <[email protected]> wrote: > On Tue, Mar 19, 2013 at 3:11 PM, Donald Stufft <[email protected]> wrote: >> >> On Mar 19, 2013, at 2:04 PM, Richard Jones <[email protected]> wrote: >> >>> Hi all, >>> >>> I present for your deliberation a draft PEP for the inclusion of a pip >>> bootstrap program in Python 3.4. Discussion of this PEP should remain >>> here on the distutils SIG. >>> >>> The PEP is revision controlled in my bitbucket account >>> https://bitbucket.org/r1chardj0n3s/pypi-pep (this is also where I'm >>> intending to develop the implementation.) >>> >>> >>> Richard >>> >>> PEP: XXXX >>> Title: Inclusion of pip bootstrap in Python installation >>> Version: >>> Last-Modified: >>> Author: Richard Jones <[email protected]> >>> BDFL-Delegate: Nick Coghlan <[email protected]> >>> Discussions-To: <[email protected]> >>> Status: Draft >>> Type: Standards Track >>> Created: 18-Mar-2013 >>> Python-Version: 3.4 >>> Post-History: 19-Mar-2013 >>> >>> Abstract >>> ======== >>> >>> This PEP proposes the inclusion of a pip boostrap executable in the Python >>> installation to simplify the use of 3rd-party modules by Python users. >>> >>> This PEP does not propose to include the pip implementation in the Python >>> standard library. Nor does it propose to implement any package management or >>> installation mechanisms beyond those provided by PEPs 470 ("The Wheel Binary >>> Package Format 1.0") and TODO distlib PEP. >>> >>> >>> Rationale >>> ========= >>> >>> Currently the user story for installing 3rd-party Python modules is >>> not as simple as it could be. It requires that all 3rd-party modules inform >>> the user of how to install the installer, typically via a link to the >>> installer. That link may be out of date or the steps required to perform the >>> install of the installer may be enough of a roadblock to prevent the user >>> from >>> further progress. >>> >>> Large Python projects which emphasise a low barrier to entry have shied away >>> from depending on third party packages because of the introduction of this >>> potential stumbling block for new users. >>> >>> With the inclusion of the package installer command in the standard Python >>> installation the barrier to installing additional software is considerably >>> reduced. It is hoped that this will therefore increase the likelihood that >>> Python projects will reuse third party software. >>> >>> It is also hoped that this is reduces the number of proposals to include >>> more and more software in the Python standard library, and therefore that >>> more popular Python software is more easily upgradeable beyond requiring >>> Python installation upgrades. >>> >>> >>> Proposal >>> ======== >>> >>> Python install includes an executable called "pip" that attempts to import >>> pip >>> machinery. If it can then the pip command proceeds as normal. If it cannot >>> it >>> will bootstrap pip by downloading the pip implementation wheel file. >>> Once installed, the pip command proceeds as normal. >>> >>> A boostrap is used in the place of a the full pip code so that we don't have >>> to bundle pip and also the install tool is upgradeable outside of the >>> regular >>> Python upgrade timeframe and processes. >>> >>> To avoid issues with sudo we will have the bootstrap default to installing >>> the >>> pip implementation to the per-user site-packages directory defined in PEP >>> 370 >>> and implemented in Python 2.6/3.0. Since we avoid installing to the system >>> Python we also avoid conflicting with any other packaging system (on Linux >>> systems, for example.) If the user is inside a virtual environment (TODO PEP >>> ref) then the pip implementation will be installed into that virtual >>> environment. >>> >>> The bootstrapping process will proceed as follows: >>> >>> 1. The user system has Python (3.4+) installed. In the "scripts" directory >>> of >>> the Python installation there is the bootstrap script called "pip". >>> 2. The user will invoke a pip command, typically "pip install <package>", >>> for >>> example "pip install Django". >>> 3. The boostrap script will attempt to import the pip implementation. If >>> this >>> succeeds, the pip command is processed normally. >>> 4. On failing to import the pip implementation the bootstrap notifies the >>> user >>> that it is "upgrading pip" and contacts PyPI to obtain the latest download >>> wheel file (see PEP 427.) >>> 5. Upon downloading the file it is installed using the distlib installation >>> machinery for wheel packages. Upon completing the installation the user >>> is notified that "pip has been upgraded." TODO how is it verified? >>> 6. The pip tool may now import the pip implementation and continues to >>> process >>> the requested user command normally. >>> >>> Users may be running in an environment which cannot access the public >>> Internet >>> and are relying solely on a local package repository. They would use the >>> "-i" >>> (Base URL of Python Package Index) argument to the "pip install" command. >>> This >>> use case will be handled by: >>> >>> 1. Recognising the command-line arguments that specify alternative or >>> additional >>> locations to discover packages and attempting to download the package >>> from those locations. >>> 2. If the package is not found there then we attempt to donwload it using >>> the standard "https://pypi.python.org/pypi/simple/pip"; index. >>> 3. If that also fails, for any reason, we indicate to the user the operation >>> we were attempting, the reason for failure (if we know it) and display >>> further instructions for downloading and installing the file manually. >>> >>> Manual installation of the pip implementation will be supported through the >>> manual download of the wheel file and "pip install <downloaded wheel file>". >>> >>> This installation will not perform standard pip installation steps of >>> saving the >>> file to a cache directory or updating any local database of installed files. >>> >>> The download of the pip implementation install file should be performed >>> securely. The transport from pypi.python.org will be done over HTTPS but >>> the CA >>> certificate check will most likely not be performed. Therefore we will >>> utilise the embedded signature support in the wheel format to validate the >>> downloaded file. >> >> Major concern is how will this deal with key revocation? If the key used to >> sign the pip wheels gets compromised what's the path for this tool to revoke >> the key? > > The wheel scheme skips revocation to simplify the implementation. You > would be hard pressed to argue that it's not better than the current > pypi security model ;-) > > A proper revocation model would look like TUF, a simple one would > consist of grabbing the author keys over HTTPS at time of use. The > scheme is flipped from the revocation model: require an up-to-date > assertion that the key is current in order to trust the key, instead > of trusting a key until a revocation happens. > >> On the side of the HTTPS I've been looking into this recently as far as >> Python goes. If openssl is correctly configured (this is the case on Linux, >> and any Python compiled against the OSX OpenSSL) then >> `urllib.request.urlopen('https://pypi.python.org/', cadefault=True) will do >> the right thing. This gets sticker on cases where openssl _isn't_ configured >> with a default set of certificates (Windows i'm assuming, Homebrew on OSX >> for sure) this will cause a certificate error. It's possible a workable >> solution can be designed via SSL.
It would also be incredibly easy to require n signatures on the wheel instead of just 1. The signature format used, JWS-JS, already holds a list of signatures. This would make it much harder to compromise the system by pwning a single pip developer's machine and may help to put your mind at ease about revocation. _______________________________________________ Distutils-SIG maillist - [email protected] http://mail.python.org/mailman/listinfo/distutils-sig
