On Wed, 26 Mar 2014 21:02:58 -0700 Maru Newby <ma...@redhat.com> wrote: > > > > If an API is made and documented properly what domain expertise > > would be needed to use it? The opposite is true for tempest and the > > tests themselves. The tempest team focuses on just tests so they > > know how to write good tests and are able to leverage common > > underlying framework code. > > Given that documentation is typically finalized only late in the > cycle, are you suggesting that we forego api testing until possibly > well after the code has been written? Plus, it is a rare api that > doesn't have to evolve in response to real-world experience with > early implementations. The sooner we can write functional api tests, > the sooner we can identify shortcomings that need to be addressed - > and the less costly they will be to fix.
So although "proper" documentation may only be finalized late in the cycle, there really should be a specification of the API and how it behaves done before the implementation is finished. At least in Nova land I think the lack of this has been a major cause of features having trouble merging and also us having flaws in both our implementation (semantic behaviour which was accidental rather than designed) and testing (incomplete coverage). Also we have API unit testing in Nova but the tempest API testing still ends up picking up more issues (perhaps because its generally not the person writing the code who ends up writing the tests). I think it also increases the chance that a backwards incompatible API changes will get picked up. Chris _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev