Thanks to all who looked at https://review.openstack.org/#/c/64733/.
There were a few minor issues I will address but the biggest one was the
suggestion to run each variation as a separate test case using
testscenarios. After looking into that I see a problem with this use
case. Many of these negative tests need to allocate resources such as
servers that are then referenced in the test cases. Our tests currently
do stuff like that in setUpClass. But testscenarios, unless I
misunderstand the code, replicates the whole class which means the
resource setup would also be duplicated for each test. I think we need
to share allocated resources across the potentially large number of
negative variations, and see two options for that:
1. Some magic similar to testscenarios but that creates test methods
with varying parameters instead of test classes with varying attributes.
Leave allocation in setUpClass.
2. Using testscenarios as-is but move the allocation/cleanup to the
module level (fixtures?)
This would be a little complicated because a bunch of the setup
stuff is defined as methods on the test base classes. Also, the classes
would
run in parallel.
Thoughts or opinions?
-David
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
