On Fri, Jun 17, 2016, at 04:16 PM, Matt Riedemann wrote: > I was reviewing this today: > > https://review.openstack.org/#/c/326940/ > > And I said to myself, 'self, do we really need to subclass the API > samples functional tests for this microversion given this change doesn't > modify the request/response body, it's only adding paging support?'. > > https://review.openstack.org/#/c/326940/6/nova/tests/functional/api_sample_tests/test_hypervisors.py > > The only change here is listing hypervisors, and being able to page on > those if the microversion is high enough. So the API samples don't > change at all, they are just running against a different microversion. > > The same goes for the REST API unit tests really: > > https://review.openstack.org/#/c/326940/6/nova/tests/unit/api/openstack/compute/test_hypervisors.py > > I'm not sure if the test subclassing is just done like this for new > microversions because it's convenient or if it's because of regression > testing - knowing that we aren't changing a bunch of other REST methods > in the process, so the subclassed tests aren't testing anything > different from the microversion that came before them. > > The thing I don't like about the test subclassing is all of the > redundant testing that goes on, and people might add tests to the parent > class not realizing it's subclassed and thus duplicating test cases with > no functional change.
I agree that the naive subclassing is wasteful. I would rather see tests that purposely check the changes not just duplicate things at a different microversion. If there's a concern about regressions I think a better approach would be to have a check against 'latest' which would catch accidental changes, not check unchanged request/reponses against microversions where they aren't intended to change. For my own curiosity I checked the timings and the duplicated tests took about 20 seconds to run. That could quickly add up to the point where a significant amount of time is spent on unnecessary testing. > > Am I just having some Friday crazies? Ultimately this doesn't hurt > anything really but thought I'd ask. > > -- > > Thanks, > > Matt Riedemann > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev