On Fri, Oct 20, 2017 at 4:32 PM, Doug Hellmann <d...@doughellmann.com> wrote:
> Excerpts from Clark Boylan's message of 2017-10-20 13:14:13 -0700: > > On Fri, Oct 20, 2017, at 11:17 AM, Clark Boylan wrote: > > > On Fri, Oct 20, 2017, at 07:23 AM, Doug Hellmann wrote: > > > > It sounds like the PyPI/PyPA folks are planning some major changes to > > > > pip internals, soon. > > > > > > > > I know pbr uses setuptools, and I don't think it uses pip, but if > > > > someone has time to verify that it would be helpful. > > > > > > > > We'll also want to watch out for breakage in normal use of pip 10. If > > > > they're making changes this big, they may miss something in their own > > > > test coverage that affects our jobs. > > > > > > > > > > After a quick skim of PBR I don't think we use pip internals anywhere. > > > Its all executed via the command itself. That said we should test this > > > so I've put up https://review.openstack.org/513825 (others should feel > > > free to iterate on it if it doesn't work) to install latest pip master > > > in a devstack run. > > > > The current issue this change is facing can be seen at > > http://logs.openstack.org/25/513825/4/check/legacy-tempest- > dsvm-py35/c31deb2/logs/devstacklog.txt.gz#_2017-10-20_20_07_54_838. > > The tl;dr is that for distutils installed packages (basically all the > > distro installed python packges) pip refuses to uninstall them in order > > to perform upgrades because it can't reliably determine where all the > > files are. I think this is a new pip 10 behavior. > > > > In the general case I think this means we can not rely on global pip > > installs anymore. This may be a good thing to bring up with upstream > > PyPA as I expect it will break a lot of people in a lot of places (it > > will break infra for example too). > > > > Clark > > > > Yes, it would be good for someone who understands the cases where we're > doing these sorts of global package installations using pip to post on > distutils-sig explaining the breakage and asking for some time to let us > work around the issue. > > We have a couple of ways to mitigate it, depending on the situation. > > 1. Pin the affected packages to the system package versions in our > constraints list. > > 2. Install into a virtualenv that ignores system packages, avoiding the > need to upgrade at all. > > 3. Remove the system package before installing from pip (why is it there > in the first place?). > > It's not clear to me which is the best approach. I've added > [devstack][qa] to the subject line to draw some more attention to > this thread from folks familiar with these jobs. > > Doug > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > I have used option 2 with great success to avoid the issue of system package conflicts for a couple of years now, back when pip would still uninstall system packages for upgrades. Option 3 is difficult to make work since so many packages have dependancies on python-* packages, I abandoned that approach fairly quickly. Thanks, SamYaple
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev