2016-03-17 5:32 GMT-07:00 Adam Young <ayo...@redhat.com>: > On 03/16/2016 11:01 PM, Ken'ichi Ohmichi wrote: >> >> 2016-03-16 19:29 GMT-07:00 Adam Young <ayo...@redhat.com>: >>> >>> On 03/16/2016 09:20 PM, Ken'ichi Ohmichi wrote: >>>> >>>> Hi >>>> >>>> I have one proposal[1] related to negative tests in Tempest, and >>>> hoping opinions before doing that. >>>> >>>> Now Tempest contains negative tests and sometimes patches are being >>>> posted for adding more negative tests, but I'd like to propose >>>> removing them from Tempest instead. >>>> >>>> Negative tests verify surfaces of REST APIs for each component without >>>> any integrations between components. That doesn't seem integration >>>> tests which are scope of Tempest. >>>> In addition, we need to spend the test operating time on different >>>> component's gate if adding negative tests into Tempest. For example, >>>> we are operating negative tests of Keystone and more >>>> components on the gate of Nova. That is meaningless, so we need to >>>> avoid more negative tests into Tempest now. >>>> >>>> If wanting to add negative tests, it is a nice option to implement >>>> these tests on each component repo with Tempest plugin interface. We >>>> can avoid operating negative tests on different component gates and >>>> each component team can decide what negative tests are valuable on the >>>> gate. >>> >>> >>> Hear hear! For Keystone, please put them in the Keystone Client >>> Functional >>> tests. >> >> They are negative tests of Keystone API, not keystoneclient tests. >> These tests are implemented with Tempest original rest clients without >> official clients because it is nice to test variable cases in Tempest. >> So it is nice to keep them in Keystone repo, I feel. > > > That works, too, and there is a Functional test section in Keystone Proper. > > Most of the Keystone APIs are fairly well covered by the Unit tests, which > make a full HTTP call through a liteish server, so the functional tests are > really to cover the live database (soon to include LDAP, hopefully) > integrations.
Nice point. Yeah, most these cases(without DB) are covered by unit tests of each project, I guess. At least, Nova also covers these cases with unit tests. Thanks __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev