On Mon, Sep 21, 2015 at 02:01:06PM +1200, Steve Baker wrote: > On 20/09/15 20:24, Qiming Teng wrote: > >Speaking of adding tests, we need hands on improving Heat API tests in > >Tempest [1]. The current test cases there is a weird combination of API > >tests, resource type tests, template tests etc. If we decide to move > >functional tests back to individual projects, some test cases may need > >to be deleted from tempest. > > > >Another important reason of adding API tests into Tempest is because > >the orchestration service is assessed [2] by the DefCore team using > >tests in Tempest, not in-tree test cases. > > > >The heat team has done a lot (and killed a lot) work to make the API as > >stable as possible. Most of the time, there would be nothing new for > >testing. The API surface tests may become nothing but waste of time if > >we keep running them for every single patch. > Thanks for raising this. Wherever they live we do need a dedicated > set of tests which ensure the REST API is fully exercised. > >So... my suggestions: > > > >- Remove unnecessary tests in Tempest; > agreed > >- Stop adding API tests to Heat locally; > >- Add API tests to Tempest instead, in an organized way. (refer to [3]) > I would prefer an alternative approach which would result in the > same end state: > - port heat_integrationtests to tempest-lib > - build a suite of REST API tests in heat_integrationtests > - work with defcore to identify which heat_integrationtests tests to > move to tempest
Sounds a reasonable approach. Thanks. Regards, Qiming > >[1] > >http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/orchestration/ > >[2] https://review.openstack.org/#/c/216983/ > >[3] https://review.openstack.org/#/c/210080/ > > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
