On Wed, 2014-02-12 at 14:17 -0800, Maru Newby wrote: > On Feb 12, 2014, at 1:59 PM, Sean Dague <s...@dague.net> wrote: > > > On 02/12/2014 04:25 PM, Maru Newby wrote: > >> > >> On Feb 12, 2014, at 12:36 PM, Sean Dague <s...@dague.net> wrote: > >> > >>> On 02/12/2014 01:48 PM, Maru Newby wrote: > >>>> At the last 2 summits, I've suggested that API tests could be maintained > >>>> in the Neutron tree and reused by Tempest. I've finally submitted some > >>>> patches that demonstrate this concept: > >>>> > >>>> https://review.openstack.org/#/c/72585/ (implements a unit test for the > >>>> lifecycle of the network resource) > >>>> https://review.openstack.org/#/c/72588/ (runs the test with tempest > >>>> rest clients) > >>>> > >>>> My hope is to make API test maintenance a responsibility of the Neutron > >>>> team. The API compatibility of each Neutron plugin has to be validated > >>>> by Neutron tests anyway, and if the tests are structured as I am > >>>> proposing, Tempest can reuse those efforts rather than duplicating them. > >>>> > >>>> I've added this topic to this week's agenda, and I would really > >>>> appreciate it interested parties would take a look at the patches in > >>>> question to prepare themselves to participate in the discussion. > >>>> > >>>> > >>>> m. > >>>> _______________________________________________ > >>>> OpenStack-dev mailing list > >>>> OpenStack-dev@lists.openstack.org > >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>>> > >>> > >>> Realistically, having API tests duplicated in the Tempest tree is a > >>> feature, not a bug. > >>> > >>> tempest/api is there for double book keep accounting, and it has been > >>> really effective at preventing accidental breakage of our APIs (which > >>> used to happen all the time), so I don't think putting API testing in > >>> neutron obviates that. > >> > >> Given how limited our testing resources are, might it be worth considering > >> whether 'double-entry accounting' is actually the best way to preventing > >> accidental breakage going forward? Might reasonable alternatives exist, > >> such as clearly separating api tests from other tests in the neutron tree > >> and giving review oversight only to qualified individuals? > > > > Our direct experience is that if we don't do this, within 2 weeks some > > project will have landed API breaking changes. This approach actually > > takes a lot of review load off the core reviewers, so reverting to a > > model which puts more work back on the review team (given the current > > review load), isn't something I think we want. > > Just so I'm clear, is there anything I could say that would change your mind?
I'd like to discuss this at tomorrow's IRC meeting, if possible. I think that the model that Maru came up with (PoC code in the two patches referenced above) are actually a pretty slick way of dealing with this issue, and along with the ongoing efforts to "libify" Tempest, I think that we should head in this direction if possible. Best, -jay _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
