Hi All,
My coworker and I would like to start filling out some gaps in api coverage
that we see in the functional integration tests. We have one patch up for
review (https://review.openstack.org/#/c/219025/). We got a comment saying that
any new stack creation will prolong the testing cycle. We agree with that and
it got us thinking about a few things -
1. We are planning on adding tests for the following api's: event api's,
template api's, software config api's, cancel stack updates, check stack
resources and show resource data. These are the api's that we saw aren't
covered in our current integration tests. Please let us know if you feel we
need tests for these upstream, if we're missing something or if it's already
covered somewhere.
2. To conserve the creation of stacks would it make sense to add one test
and then under that we could call sub methods that will run tests against that
stack. So something like this:def _test_template_apis()
def _test_softwareconfig_apis()
def _test_event_apis()
def test_event_template_softwareconfig_apis(self):
stack_id = self.stack_create(...)
self._test_template_apis(stack_id)
self._test_event_apis(stack_id)
self._test_softwareconfig_apis(stack_id)
3. The current tests are divided into two folders - scenario and functional.
To help with organization - under the functional folder, would it make sense to
add an 'api' folder, 'resource' folder and 'misc folder? Here is what we're
thinking about where each test can be put:
* API folder - test_create_update.py, test_preview.py
*
Resource folder - test_autoscaling.py, test_aws_stack.py,
test_conditional_exposure.py, test_create_update_neutron_port.py,
test_encryption_vol_type.py, test_heat_autoscaling.py, test_instance_group.py,
test_resource_group.py, test_software_config.py, test_swiftsignal_update.py
*
Misc folder - test_default_parameters.py, test_encrypted_parameter.py,
test_hooks.py, test_notifications.py, test_reload_on_sighup.py,
test_remote_stack.py, test_stack_tags.py, test_template_resource.py,
test_validation.py
4. Should we add to our README? For example, I see that we use TestResource
as a resource in some of our tests but we don't have an explanation of how to
set that up. I'd also like add explanations about the pre-testhook and
post-testhook file and how that works and what each line does/what test it's
attached to.
5. For the tests that we're working on, should we be be adding a blueprint
or task somewhere to let everybody know that we're working on it so there is no
overlap?
6. From our observations, we think it would be beneficial to add more
comments to the existing tests. For example, we could have a minimum of a
short blurb for each method. Comments?
7. Should we add a 'high level coverage' summary in our README? It could
help all of us know at a high level where we are at in terms of which resources
we have tests for and which api's, etc.
Let us know what you all think!
Thank you!
Sabeen
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
