On Fri, Mar 10, 2017 at 3:12 PM Lance Bragstad <lbrags...@gmail.com> wrote:
> On Fri, Mar 10, 2017 at 8:49 AM, Andrea Frittoli < > andrea.fritt...@gmail.com> wrote: > > > > On Fri, Mar 10, 2017 at 2:24 PM Doug Hellmann <d...@doughellmann.com> > wrote: > > Excerpts from Ghanshyam Mann's message of 2017-03-10 10:55:25 +0900: > > On Fri, Mar 10, 2017 at 7:23 AM, Lance Bragstad <lbrags...@gmail.com> > wrote: > > > > > > > > > > > On Thu, Mar 9, 2017 at 3:46 PM, Doug Hellmann <d...@doughellmann.com> > > > wrote: > > > > > >> Excerpts from Andrea Frittoli's message of 2017-03-09 20:53:54 +0000: > > >> > Hi folks, > > >> > > > >> > I'm trying to figure out what's the best approach to fade out > testing of > > >> > deprecated API versions. > > >> > We currently host in Tempest API tests for Glance API v1, Keystone > API > > >> v2 > > >> > and Cinder API v1. > > >> > > > >> > According to the guidelines for the "follow-standard-deprecation" > tag > > >> [0], > > >> > when projects that have that tag deprecate a feature: > > >> > > > >> > "Code will be frozen and only receive minimal maintenance (just so > that > > >> it > > >> > continues to work as-is)." > > >> > > > >> > I interpret this so that projects should maintain some level of > testing > > >> of > > >> > the deprecated feature, including a deprecated API version. > > >> > The QA team does not see value in testing deprecated API versions > in the > > >> > common gate jobs, so my question is what do to with those tests. > > >> > > > >> > One option is to maintain them in Tempest until the API version is > > >> removed, > > >> > and run them in dedicated project jobs. > > >> > This means that tempest would have to run those jobs as well, so > three > > >> > extra jobs, until the API version is removed. > > >> > > > >> > The other option is to move those tests out of Tempest, into the > > >> projects. > > >> > This would imply back porting them to all relevant branches as well, > > >> but it > > >> > would have the advantage of decoupling them from Tempest. It should > be > > >> no > > >> > concern from an API stability POV since the code for that API will > be > > >> > frozen. > > >> > Tests for deprecated APIs in cinder, keystone and glance are all - > as > > >> far > > >> > as I can tell - removed or deprecated from interoperability > guidelines, > > >> so > > >> > moving the tests out of Tempest would not be an issue in that sense. > > >> > > > >> > The 2nd option involves a bit more initial overhead for the removal > of > > >> > tests, but I think it would works for the best on the long term. > > >> > > > >> > There is a 3rd option as well, which is to stop running integration > > >> testing > > >> > on deprecated API versions before they are actually removed, but I > feel > > >> > that would not meet the criteria defined by the > > >> follow-standard-deprecation > > >> > tag. > > >> > > > >> > Thoughts? > > >> > > > >> > andrea > > >> > > >> Are any of those tests used by the interoperability working group > > >> (formerly DefCore)? > > >> > > >> > > > That's a good question. I was very curious about this because last I > > > checked keystone had v2.0 calls required for defcore. Looks like that > might > > > not be the case anymore [0]? I started a similar thread to this after > the > > > PTG since that was something our group talked about extensively during > the > > > deprecation session [1]. > > > > > > Yes, it seems no Volume v1 and Keystone v2 tests usage in defcore > > 2017.01.json [0]. But there are some compute tests which internally use > > glance v1 API call [2]. But on mentioned flagged action- Nova already > moved > > to v2 APIs and tempest part is pending which can be fixed to make call on > > v2 APIs only (which can be part of this work and quick). > > > > From options about deprecated APIs testing, I am with options 2 which > > really take out the load of Tempest tests maintenance and gate. > > > > But another question is about stable branch testing of those API, like > > glance v1 and identity v2 APIs are supported (not deprecated) in Mitaka. > > As Tempest is responsible of testing all stable branch behavior too, > Should > > we keep testing them till all Mitaka EOL (till APIs are in deprecated > > state in all stable branch) ? > > Excellent point. > > > As far as I can tell: > - Cinder v1 if I'm not mistaken has been deprecated in Juno, so it's > deprecated in all supported releases. > - Glance v1 has been deprecated in Newton, so it's deprecated in all > supported releases > - Keystone v2 has been deprecated in Mitaka, so testing *must* stay in > Tempest until Mitaka EOL, which is in a month from now > > We should stop testing these three api versions in the common gate > including stable branches now (except for keystone v2 on stable/mitaka > which can run for one more month). > > Are cinder / glance / keystone willing to take over the API tests and run > them in their own gate until removal of the API version? > > > Would this process consist of porting those tests directly to the project > and hooking them in like we do for tempest/devstack plugins? > > Yes, that's what I had in mind. We are going to make test.py a stable interface soon-ish, which will mean identity v2 tests only depend on tempest stable interfaces - i.e. no gate breakages. > > Doug > > > > > > > > > [0] https://github.com/openstack/defcore/blob/master/2017.01.json > > > [1] http://lists.openstack.org/pipermail/openstack-dev/ > > > 2017-March/113166.html > > > > > > [2] > > https://git.openstack.org/cgit/openstack/defcore/tree/2017.01.json#n294 > > > > > > > > > > > >> Doug > > >> > > >> > > > >> > [0] > > >> > https://governance.openstack.org/tc/reference/tags/assert_fo > > >> llows-standard-deprecation.html > > >> > > >> ____________________________________________________________ > > >> ______________ > > >> OpenStack Development Mailing List (not for usage questions) > > >> Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscrib > > >> e > > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > >> > > > > > > > > > > __________________________________________________________________________ > > > OpenStack Development Mailing List (not for usage questions) > > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev