The coverage can be expanded quite a bit by having tests that perform certain 'white-box' actions such as updating DB entries, and restoring them at the end of the test. But most of these tests would be negative, but important, scenarios. These should be avoided if they can be replaced with RESTful APIs. However, if we do need to use them we need to give them some identifier so that they do not interfere with the normal execution of Tempest blackbox tests.
Sure, we could avoid too many attrs within tempest so that we don't have to create different nose command sets with arguments for every attr combination. I'll check how we can get this done through configs. Agreed with David that the default for tempest should not be devstack. As for my specific use case to cause expired tokens, I think Daryl's suggestion to update the Keystone API is great, since it is useful to have some kind of forced disable/invalidate of tokens. Regards, Rohit -----Original Message----- From: openstack-qa-team-bounces+rohit.karajgi=nttdata....@lists.launchpad.net [mailto:openstack-qa-team-bounces+rohit.karajgi=nttdata....@lists.launchpad.net] On Behalf Of David Kranz Sent: Monday, April 30, 2012 6:28 PM To: [email protected] Subject: Re: [Openstack-qa-team] Devstack dependent tests We should have as much coverage as possible. But since we have been using config file variables to control which tests to run based on configuration (such as resize_available) it would be better to introduce a new config variable rather than making people tweak the arguments to nose. I also think the default for tempest should be that it is not using devstack. -David On 4/30/2012 12:25 AM, Karajgi, Rohit wrote: > Hi, > > We are writing new tests for keystone and some of these tests need to touch > keystone database. > I really want to avoid this, but unfortunately there are no RESTful APIs > supported in stable/essex to do the job. > > One of the example is > 1. Check if get_tenants api fails for expired token. There is no way I can > set expiry date of the token using admin RESTful API. > > So currently I'm planning to use mysql client commands and set the expiry > date. > All such tests will be put in the attr decorator with the name "devstack". So > any one who doesn't want to run such tests should run tempest with nosetests > -a kind!=devstack. > > Does it make sense to add such tests? > > Regards, > Rohit > > ______________________________________________________________________ > Disclaimer:This email and any attachments are sent in strictest > confidence for the sole use of the addressee and may contain legally > privileged, confidential, and proprietary data. If you are not the > intended recipient, please advise the sender by replying promptly to > this email and then delete and destroy this email and any attachments > without any further use, copying or forwarding > -- Mailing list: https://launchpad.net/~openstack-qa-team Post to : [email protected] Unsubscribe : https://launchpad.net/~openstack-qa-team More help : https://help.launchpad.net/ListHelp ______________________________________________________________________ Disclaimer:This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding -- Mailing list: https://launchpad.net/~openstack-qa-team Post to : [email protected] Unsubscribe : https://launchpad.net/~openstack-qa-team More help : https://help.launchpad.net/ListHelp
