Hi! Thanks - I've been wanting to kill this for a long time. Thanks for starting the discussion...
On 12/08/2013 07:26 PM, Brant Knudson wrote: > > We'd like to get the keystoneclient tests out of keystone. They're > serving a useful purpose of catching problems with non-backwards > compatible changes in keystoneclient so we still want them run. Problem > is they're running at the wrong time -- only on changes to keystone and > not changes to keystoneclient. > > The tests need to be run: > > When keystoneclient changes > - run the tests against the change > > When the tests change > - run the change against the current keystoneclient and also old clients > > When keystone changes > - run the tests against the change with current client You're talking about this: https://review.openstack.org/#/c/41931/ Which is almost ready to go. > So here's what I think we need to do to get keystone client tests out of > keystone: > > 1) Figure out where to put the tests - is it tempest or something else? Tempest. They should be moved to tempest. > 2) Write up a test and put it there > 3) Have a job that when there's a change in the tests it runs against > current client lib Don't need most of that - the generalized "test clients against stable versions" matrix is about to land - as long as tempest has your tests in the scenarios, you'll be fine. > 4) Expand the job to also run against old clients > - or is there 1 job per version? > - what versions? (keystone does master, essex-3, and 0.1.1) > - e.g. tox -e master,essex-3,0.1.1 essex and 0.1.1 are both older than dirt. We just removed folsom from the gate. I'd got ahead and remove these. > - suggest start with these versions and then consider what to use in > future OpenStack has a clear support policy - the gate infrastructure will be covering that. Done! > 5) Now we can start adding tests Tempest scenarios. > 6) Have a job that when there's a change in keystoneclient it runs > these tests against the change > 7) When there's a change in keystone, run these tests against the change Yup. Already accounted for. > 8) Copy the keystoneclient tests from keystone to the new location -- > will require some changes > 9) Remove the tests from keystone \o/ > 10) Move tests back to keystone where makes sense -- use webtest like v3 > tests > > I created an etherpad with this same info so it's easier to discuss: > https://etherpad.openstack.org/p/KeystoneTestsToTempest > > - Brant > > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev