On Sun, Dec 8, 2013 at 5:20 PM, Monty Taylor <mord...@inaugust.com> wrote:
> Hi! > > Thanks - I've been wanting to kill this for a long time. Thanks for > starting the discussion... > > On 12/08/2013 07:26 PM, Brant Knudson wrote: > > > > We'd like to get the keystoneclient tests out of keystone. They're > > serving a useful purpose of catching problems with non-backwards > > compatible changes in keystoneclient so we still want them run. Problem > > is they're running at the wrong time -- only on changes to keystone and > > not changes to keystoneclient. > > > > The tests need to be run: > > > > When keystoneclient changes > > - run the tests against the change > > > > When the tests change > > - run the change against the current keystoneclient and also old clients > > > > When keystone changes > > - run the tests against the change with current client > > You're talking about this: > > https://review.openstack.org/#/c/41931/ > > Which is almost ready to go. > > > So here's what I think we need to do to get keystone client tests out of > > keystone: > > > > 1) Figure out where to put the tests - is it tempest or something else? > > Tempest. They should be moved to tempest. > > > 2) Write up a test and put it there > > 3) Have a job that when there's a change in the tests it runs against > > current client lib > > Don't need most of that - the generalized "test clients against stable > versions" matrix is about to land - as long as tempest has your tests in > the scenarios, you'll be fine. > > > 4) Expand the job to also run against old clients > > - or is there 1 job per version? > > - what versions? (keystone does master, essex-3, and 0.1.1) > > - e.g. tox -e master,essex-3,0.1.1 > > essex and 0.1.1 are both older than dirt. We just removed folsom from > the gate. I'd got ahead and remove these. > > > - suggest start with these versions and then consider what to use in > > future > > OpenStack has a clear support policy - the gate infrastructure will be > covering that. Done! > > > 5) Now we can start adding tests > > Tempest scenarios. > > > 6) Have a job that when there's a change in keystoneclient it runs > > these tests against the change > > 7) When there's a change in keystone, run these tests against the change > > Yup. Already accounted for. > > > 8) Copy the keystoneclient tests from keystone to the new location -- > > will require some changes > > 9) Remove the tests from keystone \o/ > > 10) Move tests back to keystone where makes sense -- use webtest like v3 > > tests > > > > I created an etherpad with this same info so it's easier to discuss: > > https://etherpad.openstack.org/p/KeystoneTestsToTempest > > > > - Brant > > > > > > > > _______________________________________________ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > THANK YOU TO EVERYONE HELPING TO MAKE THIS HAPPEN! To cross-link, there was also a summit discussion on the topic: https://etherpad.openstack.org/p/icehouse-summit-qa-keystone -Dolph
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev