I just saw the plan merged in master: https://review.openstack.org/#/c/451492/ That's cool! Can we also backport the change to Ocata and maybe Newton so that we don't hit the same bug there? The backport is already up: https://review.openstack.org/#/c/456506/
Thanks, Ihar On Mon, Apr 3, 2017 at 4:06 PM, Clark Boylan <cboy...@sapwetik.org> wrote: > Hello, > > One of the major sets of issues currently affecting gate testing is > Libvirt stability. Elastic-recheck is tracking Libvirt crashes for us > and they happen frequently [0][1][2]. These issues appear to only affect > Ubuntu Xenial (and not Trusty or CentOS or Fedora) and after talking in > #openstack-nova it is clear that Libvirt isn't interested in debugging > such an old version of Libvirt (1.3.1). And while it isn't entirely > clear to me which exact version would be acceptable to them the Ubuntu > Cloud Archive (UCA) does publish a much newer Libvirt (2.5.0). > > I have pushed a change to devstack [3] to enable using UCA which pulls > in new Libvirt and mostly seems to work. I think we should consider > switching to UCA as this may fix our Libvirt problems and if it doesn't, > we will be closer to a version of Libvirt that upstream should be > willing to fix. > > This isn't the most straightfoward switch as UCA has a different repo > for each OpenStack release. libvirt-python is sensitve to the underlying > library changing; it is backward compatible but libvirt-python built > against older libvirt won't work against new libvirt. The result is a > libvirt-python wheel built on our wheel mirror does not work with UCA. > On the positive side both the OpenStack puppet modules and OpenStack > Ansible are using UCA with their deployment tooling so this should get > us closer to what people are using in the wild. > > After some thought I think my preferred method of rolling this out would > be to blacklist libvirt-python from our wheel mirror building entirely > and force installs to happen from source so that we are base Xenial and > $UCA_version independent (local testing of these builds show it only > takes a few seconds). Then have specific jobs (like devstack) explicitly > opt into the UCA repo appropriate for them (if any). This last bit is > from feedback from OpenStack Ansible that having the base images be > fairly clean is desirable, but it would also be hard to know which > version of UCA is appropriate for our Xenial images (this likely differs > based on the job). > > Now its entirely possible that newer Libvirt will be worse than current > (old) Libvirt; however, being closer to upstream should make getting > fixes easier. Would be great if those with a better understanding of > Libvirt could chime in on this if I am completely wrong here. > > Finally it is worth noting that we will get newer packages of other > software as well, most notably openvswitch will be version 2.6.1 instead > of 2.5.0. > > Any other thoughts or ideas? > > [0] http://status.openstack.org/elastic-recheck/#1646779 > [1] http://status.openstack.org/elastic-recheck/#1643911 > [2] http://status.openstack.org/elastic-recheck/#1638982 > [3] https://review.openstack.org/451492 > > Thank you, > Clark > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev