Sagi, I think Rally or Browbeat and other performance oriented solutions won't > serve our needs, because we run TripleO CI on virtualized environment with > very limited resources. Actually we are pretty close to full utilizing > these resources when deploying openstack, so very little is available for > test.
You can run Rally with any load. Including just starting 1 smallest VM. It may be useful to run a "limited edition" of API tests that maximize > coverage and don't duplicate, for example just to check service working > basically, without covering all its functionality. It will take very little > time (i.e. 5 tests for each service) and will give a general picture of > deployment success. It will cover fields that are not covered by pingtest > as well. You can actually pick few of scenarios that we have in Rally and cover most of the functionality. If you specify what exactly you want to test I can help with writing Rally Task for that. (it will use as minimum as possible resources) Best regards, Boris Pavlovic On Thu, Apr 6, 2017 at 2:38 AM, Dmitry Tantsur <dtant...@redhat.com> wrote: > On 04/05/2017 10:49 PM, Emilien Macchi wrote: > >> Greetings dear owls, >> >> I would like to bring back an old topic: running tempest in the gate. >> >> == Context >> >> Right now, TripleO gate is running something called pingtest to >> validate that the OpenStack cloud is working. It's an Heat stack, that >> deploys a Nova server, some volumes, a glance image, a neutron network >> and sometimes a little bit more. >> To deploy the pingtest, you obviously need Heat deployed in your >> overcloud. >> >> == Problems: >> >> Although pingtest has been very helpful over the last years: >> - easy to understand, it's an Heat template, like an OpenStack user >> would do to deploy their apps. >> - fast: the stack takes a few minutes to be created and validated >> >> It has some limitations: >> - Limitation to what Heat resources support (example: some OpenStack >> resources can't be managed from Heat) >> - Impossible to run a dynamic workflow (test a live migration for example) >> >> == Solutions >> >> 1) Switch pingtest to Tempest run on some specific tests, with feature >> parity of what we had with pingtest. >> For example, we could imagine to run the scenarios that deploys VM and >> boot from volume. It would test the same thing as pingtest (details >> can be discussed here). >> Each scenario would run more tests depending on the service that they >> run (scenario001 is telemetry, so it would run some tempest tests for >> Ceilometer, Aodh, Gnocchi, etc). >> We should work at making the tempest run as short as possible, and the >> close as possible from what we have with a pingtest. >> > > A lot of work is going into Tempest itself and its various plugins, so > that it becomes a convenient and universal tool to test OpenStack clouds. > While we're not quite there in terms of convenience, it's hard to match the > coverage of tempest + plugins. I'd prefer TripleO use (some subset of) > Tempest test suite(s). > > >> 2) Run custom scripts in TripleO CI tooling, called from the pingtest >> (heat template), that would run some validations commands (API calls, >> etc). >> It has been investigated in the past but never implemented AFIK. >> >> 3) ? >> > > Unless you want to duplicate all the work that goes into Tempest ecosystem > now, this is probably not a good idea. > > >> I tried to make this text short and go straight to the point, please >> bring feedback now. I hope we can make progress on $topic during Pike, >> so we can increase our testing coverage and detect deployment issues >> sooner. >> >> Thanks, >> >> > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
