On 10/08/2015 07:39 PM, Robert Collins wrote: > This is a bugbear that keeps cropping up and biting us. I'm hoping we > can figure out a permanent fix. > > The problem that occurs is the result of a few interacting things: > - requests has very very specific versions of urllib3 it works with. > So specific they aren't always released yet. > > - Linux vendors often unbundle urllib3 from requests and then apply > what patches were needed to their urllib3; while not updating their > requests package dependencies to reflect this. > > - we use urllib3 in some places and requests in others (but we don't > mix them up) > > - if for any reason we have a distro-altered requests + a > pip-installed urllib3, requests will [usually] break... see the 'not > always released yet' key thing above. > > Now, there are lots of places this last thing can happen; they all > depend on us having a dependency on requests that is compatible with > the version installed by the distro, but a urllib3 dependency that > triggers an upgrade of just urllib3. When constraints are in use, the > requests version has to match the distro requests version exactly, but > that will happen from time to time. > > e.g. > > - dvsm test jobs where the base image already has python-requests > installed in it > > - virtualenvs where system-site-packages are enabled > > > There are a few strategies that have been proposed to fix this. AIUI they are: > - make sure none of our testing environments include distro requests > packages. > > - make our requirements be tightly matched to what requests needs to > deal with the unbundling > > - teach pip how to identify and avoid this situation by always > upgrading requests (even if thats just a re-install of the version > from PyPI) when the installed requests is a distro installed version > **and** urllib3 is being modified. > > - get the distros to stop un-vendoring urllib3 > > > The first one addresses the situation for the CI gate but doesn't > avoid developers getting bitten on their local machines. And > installing any distro thing that uses requests would re-instate the > potential for breakage. So while its not harmful, I don't think its > sufficient to make this go away. > > The second is trivially insufficient - anytime requests vendored > urllib3 is not precisely identical to a released urllib3, it becomes > impossible to satisfy that via dependency version pinning - the only > way to satisfy it is with the urllib3 in the distro that has whatever > change was needed included. > > The third approach will require some negotiation I suspect - because > its aesthetically wrong: from an upstream perspective urllib3 is > independent of requests, and vice-versa, but from a distro perspective > they are tightly coupled, no variation permitted. > > The fourth approach meets the stone wall of 'but security' and 'no > redundancy permitted' - I don't have the energy to try and get through > the near-religious mindset I've encountered there before, though hey - > if Fedora and Debian and Ubuntu folk are all interested in figuring > out a sustainable way forward, that would be great: please don't feel > cut out, I'm just not expecting anything. > > If there are other approaches, great - please throw them up here. > > -Rob >
At least for my packaging for Gentoo I haven't run into any issues here. We have multiple versions of both requests and urllib3 available and I can add a missing version if needed. -- Matthew Thode (prometheanfire)
signature.asc
Description: OpenPGP digital signature
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev