On 09/03/2014 09:09 PM, Clark Boylan wrote: > > > On Wed, Sep 3, 2014, at 11:51 AM, Kuvaja, Erno wrote: >>> -----Original Message----- >>> From: Sean Dague [mailto:s...@dague.net] >>> Sent: 03 September 2014 13:37 >>> To: OpenStack Development Mailing List (not for usage questions) >>> Subject: [openstack-dev] [all] [glance] do NOT ever sort requirements.txt >>> >>> I'm not sure why people keep showing up with "sort requirements" patches >>> like - https://review.openstack.org/#/c/76817/6, however, they do. >>> >>> All of these need to be -2ed with predjudice. >>> >>> requirements.txt is not a declarative interface. The order is important as >>> pip >>> processes it in the order it is. Changing the order has impacts on the >>> overall >>> integration which can cause wedges later. >>> >>> So please stop. >>> >>> -Sean >>> >>> -- >>> Sean Dague >>> http://dague.net >>> >> >> Hi Sean & all, >> >> Could you please open this up a little bit? What are we afraid breaking >> regarding the order of these requirements? I tried to go through pip >> documentation but I could not find reason of specific order of the lines, >> references to keep the order there was 'though. >> >> I'm now assuming one thing here as I do not know if that's the case. None >> of the packages enables/disables functionality depending of what has been >> installed on the system before, but they have their own dependencies to >> provide those. Based on this assumption I can think of only one scenario >> causing us issues. That is us abusing the example in point 2 of >> https://pip.pypa.io/en/latest/user_guide.html#requirements-files meaning; >> we install package X depending on package Y>=1.0,<2.0 before installing >> package Z depending on Y>=1.0 to ensure that package Y<2.0 without >> pinning package Y in our requirements.txt. I certainly hope that this is >> not the case as depending 3rd party vendor providing us specific version >> of dependency package would be extremely stupid. >> >> Other than that I really don't know how the order could cause us issues, >> but I would be really happy to learn something new today if that is the >> case or if my assumption went wrong. >> >> Best Regards, >> Erno (jokke_) Kuvaja >> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > The issue is described in the bug that Josh linked > (https://github.com/pypa/pip/issues/988). Basically pip doesn't do > dependency resolution in a way that lets you treat requirements as order > independent. For that to be the case pip would have to evaluate all > dependencies together then install the intersection of those > dependencies. Instead it iterates over the list(s) in order and > evaluates each dependency as it is found. > > Your example basically describes where this breaks. You can both depend > on the same dependency at different versions and pip will install a > version that satisfies only one of the dependencies and not the other > leading to a failed install. However I think a more common case is that > openstack will pin a dependency and say Y>=1.0,<2.0 and the X dependency > will say Y>=1.0. If the X dependency comes first you get version 2.5 > which is not valid for your specification of Y>=1.0,<2.0 and pip fails. > You fix this by listing Y before X dependency that installs Y with less > restrictive boundaries. > > Another example of a slightly different failure would be hacking, > flake8, pep8, and pyflakes. Hacking installs a specific version of > flake8, pep8, and pyflakes so that we do static lint checking with > consistent checks each release. If you sort this list alphabetically > instead of allowing hacking to install its deps flake8 will come first > and you can get a different version of pep8. Different versions of pep8 > check different things and now the gate has broken. > > The most problematic thing is you can't count on your dependencies from > not breaking you if they come first (because they are evaluated first). > So in cases where we know order is important (hacking and pbr and > probably a handful of others) we should be listing them as early as > possible in the requirements.
So, is there a specific order to look out for? AFAIU requirements should have pbr as first requirement and test-requirements should have hacking as first one. Is there anything else? What's the best place to document this? Andreas -- Andreas Jaeger aj@{suse.com,opensuse.org} Twitter/Identica: jaegerandi SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn,Jennifer Guild,Felix Imendörffer,HRB16746 (AG Nürnberg) GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126 _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev