Donald linked to a pip bug later in this thread, so we might be able to help by working on a fix. I haven't investigated that, but I assume if it was easy the pypa team would have already fixed it.
When you saw the problem, were you running a current version of devstack with Sean's change to install oslo libraries via "pip" instead of "pip -e"? Doug On Mon, Apr 7, 2014 at 4:34 PM, Vishvananda Ishaya <vishvana...@gmail.com> wrote: > I dealt with this myself the other day and it was a huge pain. That said, > changing all the packages seems like a nuclear option. Is there any way > we could change python that would make it smarter about searching multiple > locations for namespace packages? > > Vish > > On Apr 7, 2014, at 12:24 PM, Doug Hellmann <doug.hellm...@dreamhost.com> > wrote: > >> Some of the production Oslo libraries are currently being installed >> into the "oslo" namespace package (oslo.config, oslo.messaging, >> oslo.vmware, oslo.rootwrap, and oslo.version). Over the course of the >> last 2 release cycles, we have seen an increase in the number of >> developers who end up with broken systems, where an oslo library (most >> often oslo.config) cannot be imported. This is usually caused by >> having one copy of a library installed normally (via a system package >> or via pip) and another version in "development" (a.k.a., "editable") >> mode as installed by devstack. The symptom is most often an error >> about importing oslo.config, although that is almost never the library >> causing the problem. >> >> We have already worked around this issue with the non-production >> libraries by installing them into their own packages, without using >> the namespace (oslotest, oslosphinx, etc.). We have also changed the >> way packages are installed in nova's tox.ini, to force installation of >> packages into the virtualenv (since exposing the global site-packages >> was a common source of the problem). And very recently, Sean Dague >> changed devstack to install the oslo libraries not in editable mode, >> so that installing from source should replace any existing installed >> version of the same library. >> >> However, the problems seem to persist, and so I think it's time to >> revisit our decision to use a namespace package. >> >> After experimenting with non-namespace packages, I wasn't able to >> reproduce the same import issues. I did find one case that may cause >> us some trouble, though. Installing a package and then installing an >> editable version from source leaves both installed and the editable >> version appears first in the import path. That might cause surprising >> issues if the source is older than the package, which happens when a >> devstack system isn't updated regularly and a new library is released. >> However, surprise due to having an old version of code should occur >> less frequently than, and have less of an impact than, having a >> completely broken set of oslo libraries. >> >> We can avoid adding to the problem by putting each new library in its >> own package. We still want the Oslo name attached for libraries that >> are really only meant to be used by OpenStack projects, and so we need >> a naming convention. I'm not entirely happy with the "crammed >> together" approach for oslotest and oslosphinx. At one point Dims and >> I talked about using a prefix "oslo_" instead of just "oslo", so we >> would have "oslo_db", "oslo_i18n", etc. That's also a bit ugly, >> though. Opinions? >> >> Given the number of problems we have now (I help about 1 dev per week >> unbreak their system), I think we should also consider renaming the >> existing libraries to not use the namespace package. That isn't a >> trivial change, since it will mean updating every consumer as well as >> the packaging done by distros. If we do decide to move them, I will >> need someone to help put together a migration plan. Does anyone want >> to volunteer to work on that? >> >> Before we make any changes, it would be good to know how bad this >> problem still is. Do developers still see issues on clean systems, or >> are all of the problems related to updating devstack boxes? Are people >> figuring out how to fix or work around the situation on their own? Can >> we make devstack more aggressive about deleting oslo libraries before >> re-installing them? Are there other changes we can make that would be >> less invasive? >> >> Doug >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev