Eric, There are Gorka's patches [10] to remove API Races
[10] https://review.openstack.org/#/q/project:openstack/cinder+branch:master+topic:fix/api-races-simplified Regards, Ivan Kolodyazhny, http://blog.e0ne.info/ On Wed, Mar 2, 2016 at 4:27 PM, Eric Harney <ehar...@redhat.com> wrote: > On 03/02/2016 06:25 AM, Ivan Kolodyazhny wrote: > > Hi Team, > > > > Here are my thoughts and proposals how to make Cinder testing process > > better. I won't cover "3rd party CI's" topic here. I will share my > opinion > > about current and feature jobs. > > > > > > Unit-tests > > > > - Long-running tests. I hope, everybody will agree that unit-tests > must > > be quite simple and very fast. Unit tests which takes more than 3-5 > seconds > > should be refactored and/or moved to 'integration' tests. > > Thanks to Tom Barron for several fixes like [1]. IMO, we it would be > > good to have some hacking checks to prevent such issues in a future. > > > > - Tests coverage. We don't check it in an automatic way on gates. > > Usually, we require to add some unit-tests during code review > process. Why > > can't we add coverage job to our CI and do not merge new patches, with > > will decrease tests coverage rate? Maybe, such job could be voting in > a > > future to not ignore it. For now, there is not simple way to check > coverage > > because 'tox -e cover' output is not useful [2]. > > > > > > Functional tests for Cinder > > > > We introduced some functional tests last month [3]. Here is a patch to > > infra to add new job [4]. Because these tests were moved from > unit-tests, I > > think we're OK to make this job voting. Such tests should not be a > > replacement for Tempest. They even could tests Cinder with Fake Driver to > > make it faster and not related on storage backends issues. > > > > > > Tempest in-tree tests > > > > Sean started work on it [5] and I think it's a good idea to get them in > > Cinder repo to run them on Tempest jobs and 3-rd party CIs against a real > > backend. > > > > > > Functional tests for python-brick-cinderclient-ext > > > > There are patches that introduces functional tests [6] and new job [7]. > > > > > > Functional tests for python-cinderclient > > > > We've got a very limited set of such tests and non-voting job. IMO, we > can > > run them even with Cinder Fake Driver to make them not depended on a > > storage backend and make it faster. I believe, we can make this job > voting > > soon. Also, we need more contributors to this kind of tests. > > > > > > Integrated tests for python-cinderclient > > > > We need such tests to make sure that we won't break Nova, Heat or other > > python-cinderclient consumers with a next merged patch. There is a thread > > in openstack-dev ML about such tests [8] and proposal [9] to introduce > them > > to python-cinderclient. > > > > > > Rally tests > > > > IMO, it would be good to have new Rally scenarios for every patches like > > 'improves performance', 'fixes concurrency issues', etc. Even if we as a > > Cinder community don't have enough time to implement them, we have to ask > > for them in reviews, openstack-dev ML, file Rally bugs and blueprints if > > needed. > > > > Are there any recent examples of a fix like this recently where it would > seem like a reasonable task to write a Rally scenario along with the patch? > > Not being very familiar with Rally (as I think most of us aren't), I'm > having a hard time picturing this. > > > > > [1] https://review.openstack.org/#/c/282861/ > > [2] http://paste.openstack.org/show/488925/ > > [3] https://review.openstack.org/#/c/267801/ > > [4] https://review.openstack.org/#/c/287115/ > > [5] https://review.openstack.org/#/c/274471/ > > [6] https://review.openstack.org/#/c/265811/ > > [7] https://review.openstack.org/#/c/265925/ > > [8] > > > http://lists.openstack.org/pipermail/openstack-dev/2016-March/088027.html > > [9] https://review.openstack.org/#/c/279432/ > > > > > > Regards, > > Ivan Kolodyazhny, > > http://blog.e0ne.info/ > > > > > > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev