On Fri, Feb 21, 2014 at 10:17 AM, Sean Dague <s...@dague.net> wrote: > On 02/21/2014 11:02 AM, Daniel P. Berrange wrote: >> On Fri, Feb 21, 2014 at 10:46:22AM -0500, Sean Dague wrote: >>> On 02/21/2014 09:45 AM, Daniel P. Berrange wrote: >>>> On Thu, Feb 20, 2014 at 02:45:03PM -0500, Sean Dague wrote: >>>>> >>>>> So I'm one of the first people to utter "if it isn't tested, it's >>>>> probably broken", however I also think we need to be realistic about the >>>>> fact that if you did out the permutations of dependencies and config >>>>> options, we'd have as many test matrix scenarios as grains of sand on >>>>> the planet. >>>>> >>>>> I do think in some ways this is unique to OpenStack, in that our >>>>> automated testing is head and shoulders above any other Open Source >>>>> project out there, and most proprietary software systems I've seen. >>>>> >>>>> So this is about being pragmatic. In our dependency testing we are >>>>> actually testing with most recent versions of everything. So I would >>>>> think that even with libvirt, we should err in that direction. >>>> >>>> I'm very much against that, because IME, time & time again across >>>> all open source projects I've worked on, people silently introduce >>>> use of features/apis that only exist in newer versions without anyone >>>> ever noticing until it is too late. >>>> >>>>> That being said, we also need to be a little bit careful about taking >>>>> such a hard line about "supported vs. not" based on only what's in the >>>>> gate. Because if we did the following things would be listed as >>>>> unsupported (in increasing level of ridiculousness): >>>>> >>>>> * Live migration >>>>> * Using qpid or zmq >>>>> * Running on anything other than Ubuntu 12.04 >>>>> * Running on multiple nodes >>>>> >>>>> Supported to me means we think it should work, and if it doesn't, it's a >>>>> high priority bug that will get fixed quickly. Testing is our sanity >>>>> check. But it can't be considered that it will catch everything, at >>>>> least not before the heat death of the universe. >>>> >>>> I agree we should be pragmatic here to some extent. We shouldn't aim to >>>> test every single intermediate version, or every possible permutation of >>>> versions - just a representative sample. Testing both lowest and highest >>>> versions is a reasonable sample set IMHO. >>> >>> Testing lower bounds is interesting, because of the way pip works. That >>> being said, if someone wants to take ownership of building that job to >>> start as a periodic job, I'm happy to point in the right direction. Just >>> right now, it's a lower priority item than things like Tempest self >>> testing, Heat actually gating, Neutron running in parallel, Nova API >>> coverage. >> >> If it would be hard work to do it for python modules, we can at least >> not remove the existing testing of an old libvirt version - simply add >> an additional test with newer libvirt. > > Simply adding a test with newer libvirt isn't all that simple at the end > of the day, as it requires building a new nodepool image. Because > getting new libvirt in the existing test environment means cloud > archive, and cloud archive means a ton of other new code as well. Plus > in Juno we're presumably going to jump to 14.04 as our test base, which > is going to be it's own big transition. > > So, I'm not opposed, but I also think bifurcating libvirt testing is a > big enough change in the pipeline that it needs some pretty dedicated > folks looking at it, and the implications there in. This isn't just a > yaml change, set and forget it. And given where we are in the > development cycle, I'm not sure trying to keep the gate stable with a > new libvirt which we've known to be problematic, is the right time to do > this. > > But, if someone is stepping up to work through it, can definitely mentor > them on the right places to be poking.
So it sounds like the consensus here is: * We should have a uniform policy (unless we take the platform vs app distinction) * Long term we want to have a lower bound gate job as well, but that no one has stepped up to work on it yet * Setting up libvirt min and libvirt max tests is non-trivial and needs someone to work on it So in the short term we shouldn't be forced to to hold libvirt back to the minimal supported version in gate(https://blueprints.launchpad.net/nova/+spec/support-libvirt-1x), hopefully while someone steps up to get a minimal libvirt (and python deps?) job in the gate. > > -Sean > > -- > Sean Dague > Samsung Research America > s...@dague.net / sean.da...@samsung.com > http://dague.net > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev