Bug#744145: [Python-modules-team] Bug#744145: pip3 breaks after upgrading requests
On 2015-01-26 10:25, Chris Kuehl wrote: > I wonder whether such a change should at least be documented in the > release notes, even if we can't address it because of the freeze? Yes, good idea. This would allow users to remove such libraries first and then upgrade Debian. Would you file a bug against release-notes, please? -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#744145: [Python-modules-team] Bug#744145: pip3 breaks after upgrading requests
Hi Stefano, On Mon, Jan 26, 2015 at 05:12:42AM +0200, Stefano Rivera wrote: > I don't think I consider this bug to be RC. Debian packages have > declared dependencies on other Debian packages. Replacing one with > something newer from upstream, is quite likely to break things. Thanks for responding. I do understand your reasoning behind not considering the bug for jessie. For the sake of documenting this bug better, I probably should have explained my reasoning a bit clearer. I'm afraid that the impact of the bug, particularly on upgrades, is likely to be pretty widespread: * On wheezy, if someone installs requests to system site-packages, pip will work fine. After an upgrade to jessie, pip is broken and cannot be fixed without rm-ing the appropriate directory (pip uninstall won't work, either). * As far as I'm aware, vendorizing (and on Debian, de-vendorizing) is new with the version of pip included with jessie, so the behavior that installing a different requests version (or some other devendorized library, such as colorama) can permanently break pip is new. * The default option is to install system-wide (i.e. --user is not implicit for non root, #725848), and site-packages installs are still very common, especially when one desires to have a Python binary packaged on PyPI installed system-wide. It's very easy for a user to back themselves into a corner, especially given that once requests has been installed, there is no easy or obvious way to fix pip. I wonder whether such a change should at least be documented in the release notes, even if we can't address it because of the freeze? > "sudo pip" on a Debian box is dangerous, don't do that, and rather use > virtualenvs, if you need to go off the beaten track. I agree with this, but I'm afraid that it's still a very common practice. It's not hard to find articles advising users to run `pip` as root, and I suspect that such recommendations will be the first result when an unsuspecting user searches pip errors on Google. Thanks and happy Monday, Chris -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#744145: [Python-modules-team] Bug#744145: pip3 breaks after upgrading requests
Control: severity -1 wishlist Hi Chris (2015.01.23_11:34:16_+0200) > On a clean jessie or sid system (does not happen on wheezy): > > root# apt-get install python3 python3-pip > root# pip3 install --upgrade requests > > After this, every invocation of pip3 fails: I don't think I consider this bug to be RC. Debian packages have declared dependencies on other Debian packages. Replacing one with something newer from upstream, is quite likely to break things. Supporting user-installed python libraries falls far into wishlist, for me. At this point in the freeze, I don't think it's worth bothering the release team, to fix this. "sudo pip" on a Debian box is dangerous, don't do that, and rather use virtualenvs, if you need to go off the beaten track. SR -- Stefano Rivera http://tumbleweed.org.za/ +1 415 683 3272 -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#744145: pip3 breaks after upgrading requests
Hi again maintainer,
As Yannick suggests, this seems to have been caused by this change to requests
back in July:
https://github.com/kennethreitz/requests/commit/47d0517d66e8cf5832768262221f0357ae134ad1
Now that requests.compat no longer imports IncompleteRead, it won't be
available for pip to use.
There are two sequential commits in pip which remove its use of
IncompleteRead:
https://github.com/pypa/pip/commit/53eab2357e4425436b51e109bfaedcb9f3772072
https://github.com/pypa/pip/commit/e77c0c573c059bb4a86b2c5c06835f57e4af6c47
These were merged as part of the PR: https://github.com/pypa/pip/pull/2024
Unfortunately, the commits themselves don't contain much context for the
change. It looks like it was perhaps made to satisfy CI warnings?
On Tue, Oct 21, 2014 at 10:14:49PM +0200, Yannick Roehlly wrote:
> A quick fix would be to use "from httplib import IncompleteRead" in the
> de-vendorize.patch (even if the IncompleteRead will not be raised by requests
> any more).
I've attached a patch that does this (with a minor change to support Python3),
and tested that it works.
I've also attached a patch which backports the two pip commits into the current
python-pip source package.
Both of these fix the current problem, and allow pip to work with both the
version of requests the Debian package, and the current version from PyPI,
which is likely to be installed by users. I'm not sure which is preferable, so
I've included both (only one should be applied, of course).
> Note that there may be other problems with pip using the upgraded requests.
This is probably also worth reiterating. I wonder how hard it would be to
change system pip to always use system versions of libraries. There's no
guarantee that a change to the internals of requests doesn't break pip again
tomorrow.
Thanks again!
Chris
On Fri, Jan 23, 2015 at 01:34:16AM -0800, Chris Kuehl wrote:
> Hi maintainer,
>
> I am able to reproduce this reliably.
>
> On a clean jessie or sid system (does not happen on wheezy):
>
> root# apt-get install python3 python3-pip
> root# pip3 install --upgrade requests
>
> After this, every invocation of pip3 fails:
>
> root# pip3
> Traceback (most recent call last):
> File "/usr/bin/pip3", line 9, in
> load_entry_point('pip==1.5.6', 'console_scripts', 'pip3')()
> File "/usr/lib/python3/dist-packages/pkg_resources.py", line 356, in
> load_entry_point
> return get_distribution(dist).load_entry_point(group, name)
> File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2476, in
> load_entry_point
> return ep.load()
> File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2190, in
> load
> ['__name__'])
> File "/usr/lib/python3/dist-packages/pip/__init__.py", line 61, in
>
> from pip.vcs import git, mercurial, subversion, bazaar # noqa
> File "/usr/lib/python3/dist-packages/pip/vcs/mercurial.py", line 9, in
>
> from pip.download import path_to_url
> File "/usr/lib/python3/dist-packages/pip/download.py", line 25, in
>
> from requests.compat import IncompleteRead
> ImportError: cannot import name 'IncompleteRead'
>
> The version of requests which the upgrade installs is currently 2.5.1.
>
> This renders pip unusable for many cases, and can break not just pip but
> also other Python packages. In particular, any Python package that
> depends on requests which you upgrade using system pip will permanently
> break pip.
>
> Thanks for looking into this -- I'm more than happy to help with testing
> any fixes!
>
> Happy Friday,
> Chris
From: Chris Kuehl
Date: Fri, 23 Jan 2015 14:54:24 -0800
Subject: [PATCH] Don't raise IncompleteRead if response reads fails
This is a backport of the following commits onto the python-pip package.
https://github.com/pypa/pip/commit/53eab2357e4425436b51e109bfaedcb9f3772072
https://github.com/pypa/pip/commit/e77c0c573c059bb4a86b2c5c06835f57e4af6c47
This fixes Debian bug#744145 by allowing pip to continue to work with
upgraded versions of the requests library.
---
pip/download.py | 35 +++
1 file changed, 27 insertions(+), 8 deletions(-)
diff --git a/pip/download.py b/pip/download.py
index adff2ec..768e7ba 100644
--- a/pip/download.py
+++ b/pip/download.py
@@ -22,8 +22,7 @@ from pip.log import logger
import requests, six
from requests.adapters import BaseAdapter
from requests.auth import AuthBase, HTTPBasicAuth
-from requests.compat import IncompleteRead
-from requests.exceptions import InvalidURL, ChunkedEncodingError
+from requests.exceptions import InvalidURL
from requests.models import Response
from requests.structures import CaseInsensitiveDict
@@ -416,12 +415,32 @@ def _download_url(resp, link, temp_location):
def resp_read(chunk_size):
try:
# Special case for urllib3.
-try:
-for chunk in resp.raw.stream(
-
Bug#744145: pip3 breaks after upgrading requests
Hi maintainer,
I am able to reproduce this reliably.
On a clean jessie or sid system (does not happen on wheezy):
root# apt-get install python3 python3-pip
root# pip3 install --upgrade requests
After this, every invocation of pip3 fails:
root# pip3
Traceback (most recent call last):
File "/usr/bin/pip3", line 9, in
load_entry_point('pip==1.5.6', 'console_scripts', 'pip3')()
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 356, in
load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2476, in
load_entry_point
return ep.load()
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2190, in load
['__name__'])
File "/usr/lib/python3/dist-packages/pip/__init__.py", line 61, in
from pip.vcs import git, mercurial, subversion, bazaar # noqa
File "/usr/lib/python3/dist-packages/pip/vcs/mercurial.py", line 9, in
from pip.download import path_to_url
File "/usr/lib/python3/dist-packages/pip/download.py", line 25, in
from requests.compat import IncompleteRead
ImportError: cannot import name 'IncompleteRead'
The version of requests which the upgrade installs is currently 2.5.1.
This renders pip unusable for many cases, and can break not just pip but
also other Python packages. In particular, any Python package that
depends on requests which you upgrade using system pip will permanently
break pip.
Thanks for looking into this -- I'm more than happy to help with testing
any fixes!
Happy Friday,
Chris
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
