On Sat, Sep 03, 2016 at 07:33:42PM +0200, Andreas Jaeger wrote: > Reviewing all the constraints work, I see that repositories have created > some workaround around requirements install for one of these two legimit > reasons - most often using tools/tox_install.sh from tox.ini for it: > > 1) The repository is a dependency of another one and uses constraints, > so edit upper-constraints file and allow git install. > > Example: > http://git.openstack.org/cgit/openstack/ironic-lib/tree/tools/tox_install.sh
We had a very brief discussion about this in Barcelona. The idea being to create an "incubator" style script in openstack/requirements that can be the canonical source and easily copied out to repos that need it. I'm not proposing auto syncing but it would be pretty easy to script. We need "all projects"[1] to support constraints real soon now. It seems like the majority of projects that currently do not use constraints are because they're also listed in constraints. I started looking at this today using [2] as the base in the oslo.messaging repo The good thing about this is it doesn't "just work" I hit the following problem: --- cmdargs: ['/home/stack/projects/openstack/openstack/oslo.messaging/tools/tox_install.sh', 'https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt', '/home/stack/projects/openstack/openstack/oslo.messaging/.tox/dist/oslo.messaging-5.12.1.dev10.zip'] <snip> Processing ./.tox/dist/oslo.messaging-5.12.1.dev10.zip Could not satisfy constraints for 'oslo.messaging': installation from path or url cannot be constrained to a version --- This is because we use 'edit-constraints' to change "oslo.messaging===5.12.0" to "-e file:///home/stack/projects/openstack/openstack/oslo.messaging#egg=oslo.messaging"[3] Which doesn't match because we're installing from an sdist. For development installs like this what we really want is a way to say constrain all my requirements but allow this library to be unconstrained don't we? That seems to me what we're saying in [3]. When I'm working locally I do something like: --- pip install -c https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt \ -r requirements.txt -r test-requirements.txt pip install . --- This is all leading me to think that we should just remove the constraint on $library which can be done with something like: --- [4] #!/usr/bin/env bash # Client constraint file contains this client version pin that is in conflict # with installing the client from source. We should remove the version pin in # the constraints file before applying it for from-source installation. BRANCH_NAME=XXX CLIENT_NAME=XXX set -e CONSTRAINTS_FILE=$1 shift localfile="${VIRTUAL_ENV}/upper-constraints.txt" if [[ $CONSTRAINTS_FILE != http* ]]; then CONSTRAINTS_FILE=file://$CONSTRAINTS_FILE fi curl $CONSTRAINTS_FILE -k -o $localfile sed -i~ -e "/^${CLIENT_NAME}===/d" $localfile pip install -U -c$localfile $* --- Using openstack_requirements is the "right" thing to do and we could still use edit-constraints $localfile -- $CLIENT_NAME "" do delete the entry but it seems like wasted work to me > 2) Install a package that is not on pypi. This is typical neutron or > horizon. > > Example: > http://git.openstack.org/cgit/openstack/neutron-lbaas/tree/tools/tox_install.sh This is much less thought out but can't we enhance the deps in tox.ini like: --- deps = os:neutron os:neutron-lbaas -r requirements.txt --- and then use something like: --- [5] #!/usr/bin/env bash function install_os_dep() { set -- $( echo $1 | sed -e 's/:/ /') prefix=$1 # ignored project=$2 # Now basically turn http://git.openstack.org/cgit/openstack/neutron-lbaas/tree/tools/tox_install.sh into a shell function } # Client constraint file contains this client version pin that is in conflict # with installing the client from source. We should remove the version pin in # the constraints file before applying it for from-source installation. BRANCH_NAME=XXX CLIENT_NAME=XXX set -e CONSTRAINTS_FILE=$1 shift localfile="${VIRTUAL_ENV}/upper-constraints.txt" if [[ $CONSTRAINTS_FILE != http* ]]; then CONSTRAINTS_FILE=file://$CONSTRAINTS_FILE fi curl $CONSTRAINTS_FILE -k -o $localfile sed -i~ -e "/^${CLIENT_NAME}===/d" $localfile deps="" while [ $# -gt 1 ] ; do case "$1" in os:*) install_os_dep "$1" ;; *) deps+="$1" ;; shift 1 done pip install -U -c$localfile $deps exit $? --- [6] We also discussed that the right place to fix these issues could be in tox and/or pip but as I said earlier it's a matter of balancing right vs right now :( Yours Tony. [1] For now that means all deliverables listed in openstack/governance:reference/projects.yaml, but nothing is stopping adoption in other stackforge projects. [2] http://git.openstack.org/cgit/openstack/python-neutronclient/tree/tools/tox_install.sh [3] http://git.openstack.org/cgit/openstack/python-neutronclient/tree/tools/tox_install.sh#n52 [4] https://gist.github.com/tbreeds/09c87ff4e08b2095712ea5ac4070165b#file-pip_wrap-sh [5] https://gist.github.com/tbreeds/285269c8ed35a5378a27ee7af408d670#file-pip_wrap-sh [6] Without syntax errors, and with slight re-factoring
signature.asc
Description: PGP signature
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev