Re: [openstack-dev] [Openstack-operators] [all][qa][gabbi][rally][tempest] Extend "rally verfiy" to unify work with Gabbi, Tempest and all in-tree functional tests

2015-03-19 Thread Andrey Kurilin
I have an alternative solution for `rally verify start` command. What do you think about similar management stuff for verifiers as we have for deployments? It requires several changes: - `rally verifiers` command: Implementation of new command, which will manage(install, reinstall, remove, confi

Re: [openstack-dev] [Openstack-operators] [all][qa][gabbi][rally][tempest] Extend "rally verfiy" to unify work with Gabbi, Tempest and all in-tree functional tests

2015-03-11 Thread Boris Pavlovic
Alex, * rally plugin should be a part of project (for example, located in > functional tests directory) There are 2 issues with such solution: 1) If rally didn't load plugin, command "rally verify " won't exist 2) Putting some strange Rally plugin to source of other projects will be quit

Re: [openstack-dev] [Openstack-operators] [all][qa][gabbi][rally][tempest] Extend "rally verfiy" to unify work with Gabbi, Tempest and all in-tree functional tests

2015-03-11 Thread Andrey Kurilin
> $ rally verify https://github.com/openstack/nova start As one of end-users of Rally, I dislike such construction, because I don't want to remember links to repos, they are too long for me:) On Wed, Mar 11, 2015 at 12:49 PM, Aleksandr Maretskiy < amarets...@mirantis.com> wrote: > The idea is gr

Re: [openstack-dev] [Openstack-operators] [all][qa][gabbi][rally][tempest] Extend "rally verfiy" to unify work with Gabbi, Tempest and all in-tree functional tests

2015-03-11 Thread Aleksandr Maretskiy
The idea is great, but IMHO we can move all project-specific code out of rally, so: * rally plugin should be a part of project (for example, located in functional tests directory) * use {project url} instead of {project name} in rally verify command, example: $ rally verify https://gi

Re: [openstack-dev] [Openstack-operators] [all][qa][gabbi][rally][tempest] Extend "rally verfiy" to unify work with Gabbi, Tempest and all in-tree functional tests

2015-03-10 Thread Timur Nurlygayanov
Hi, I like this idea, we use Rally for OpenStack clouds verification at scale and it is the real issue - how to run all functional tests from each project with the one script. If Rally will do this, I will use Rally to run these tests. Thank you! On Mon, Mar 9, 2015 at 6:04 PM, Chris Dent wrote

Re: [openstack-dev] [Openstack-operators] [all][qa][gabbi][rally][tempest] Extend "rally verfiy" to unify work with Gabbi, Tempest and all in-tree functional tests

2015-03-09 Thread Chris Dent
On Mon, 9 Mar 2015, Davanum Srinivas wrote: 2. Is there a "test" project with Gabbi based tests that you know of? In addition to the ceilometer tests that Boris pointed out gnocchi is using it as well: https://github.com/stackforge/gnocchi/tree/master/gnocchi/tests/gabbi 3. What changes

Re: [openstack-dev] [Openstack-operators] [all][qa][gabbi][rally][tempest] Extend "rally verfiy" to unify work with Gabbi, Tempest and all in-tree functional tests

2015-03-09 Thread Boris Pavlovic
Davanum, 1. Suppose a project say Nova wants to enable this Rally Integration > for its functional tests, what does that project have to do? (other > than the existing well defined tox targets) Actually project by it self shouldn't do anything. The whole work regarding to integration belongs t

Re: [openstack-dev] [Openstack-operators] [all][qa][gabbi][rally][tempest] Extend "rally verfiy" to unify work with Gabbi, Tempest and all in-tree functional tests

2015-03-09 Thread Davanum Srinivas
Boris, 1. Suppose a project say Nova wants to enable this Rally Integration for its functional tests, what does that project have to do? (other than the existing well defined tox targets) 2. Is there a "test" project with Gabbi based tests that you know of? 3. What changes if any are needed in Gab