On 1/24/2015 12:13 PM, Sean Dague wrote:
I've been looking at the following patch series - https://review.openstack.org/#/c/131691/13 for removing database requirements from some tests. I whole heartedly support getting DB usage out of tests, but I'd like to make sure that we don't create new challenges in the process. The conditional create_db parameter in test functions adds a bit more internal test complexity than I think we should have. I'd like to propose instead DB usage should be removed per test Class as an atomic unit. If that turns into too large a patch that probably means breaking apart the test class into smaller test classes first. The other thing to be careful in understanding the results of timing tests. The way the database fixture works it caches the migration process - https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L206 That actually means that the overhead of the db-migration sync is paid only once per testr worker (it's 1s on my fast workstation, might be 2s on gate nodes). The subsequence db setups for tests 2 -> N in the worker only take about 0.020s on my workstation (scale appropriately). So removing all the unneeded db setup code is probably only going to save ~30s over an entire test run. Which doesn't mean it shouldn't be done, there are other safety reasons we shouldn't let every test randomly punch data into the db and still pass. But time savings should not be the primary motivator here, because it's actually not nearly as much gain as it looks like from running only a small number of tests. -Sean __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Big +1 to doing this on a class by class basis, however, that can get ugly fast [1].
I commented on the change for test_compute to just move those to test_compute_api/mgr as appropriate, we should be freezing test_compute for the most part unless you're just modifying an existing test for some small wrinkle in the code.
I don't think we can completely change test_compute over to no-db in one change, that would be huge, we should just work on moving pieces over to those other modules that use NoDBTestCase (or move the bigger functional-type tests into nova.tests.functional).
One candidate is moving the big shelve/unshelve tests into test_compute_mgr/api, those have been a latent source of race bugs in the gate because of weird DB interactions/expectations/assumptions being borked by other tests running at the same time hitting the DB (because of so many different test classes that extend test_compute.BaseTestCase).
[1] https://review.openstack.org/#/c/142655/ -- Thanks, Matt Riedemann __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
