On Wed, Sep 10, 2014 at 5:42 AM, Alex Brett <alex.br...@citrix.com> wrote: > Hello all, > > At the moment we have a lot of Marvin tests that follow a pattern that looks > roughly like this: > > 1. Setup some resources (e.g. accounts, service offerings, VMs etc) > 2. Add the resources to a list in the testcase (often called self.cleanup) > 3. Do the test(s) > 4. Call cleanup_resources with the list of resources from 2 > > (obviously in some cases resources get created/allocated during the actual > test rather than in setup, but it's a similar principle) > > In theory this is fine, however there are a number of cases where resources > are being created and then not added to the cleanup list, which results in > things being 'left behind', potentially using up resources which may then > affect future tests. For example I'm currently attempting to run various > tests in parallel (to speed up execution), and I'm hitting some issues I > believe to be caused by this. > > The thought that occurs to me here is do we actually need the testcase to > have to manually add resources to a list to cleanup, with the inherent risk > of resources getting missed etc - could we not make this something the > framework does for us (at least by default, with the option to override the > behaviour if needed). > > I've got some ideas as to how this could be done (one example that's a bit of > a layer violation but might be acceptable would be to wrap/extend the > apiClient to have a method that can be called on it from the various object > create methods to add the resulting object for cleanup), but before I go > ahead and start trying to prototype something I wanted to see if anybody had > any reasons why this sort of automatic cleanup behaviour might be a bad idea > or has investigated anything similar in the past? >
+1 - the cleanup of cloud resources is nasty especially when VMs and billable entities are left behind. Many tests that follow fail as a result of exhausted capacity. The nose framework sort of imposes the setUp - test - tearDown model of test case authoring and perhaps led us down that path of accumulating entities in cleanup lists. It would be elegant if you can identify the corresponding Object.destroy method and add the necessary finalizer automatically during create. Sort of like the way unittest2 and py.test do it. The ACS API is quite expressive and broken down easily into this model where you can figure out the entity/resource you operate upon. Each entity has its API for creation and a corresponding one for listing and destroy/deletion. If that's too involved the overloaded apiClient works fine too. > Cheers, > Alex >
