Re: [Openstack-operators] regression testing before upgrade (operators view)

[George Shuklin]

Hello.

Thank you for reply.

In those days I've tested tempest and rally. I found rally more suited 
for my goals. Insofar I'd like to admit that it's too overloaded and 
when I start poking around I quickly stuck with 'verify' mode, which in 
turn stuck in our neutron configuration (we have no tenant-allocated 
networks, only public ones).


But after I found rally task start, and mangled samples a bit to fit to 
our needs, it really, really extremely close to what we want to have. I 
was disappointed by Ubuntu packaging (as usual) - their rally package 
broken and does not create any entry points at all. It worked much 
better in venv environment.


Probably I'll try to combine jenkins, grafana, kibana and rally 
together, may be even for periodic service validation.


Thank you for advice.

On 08/29/2017 11:34 PM, Boris Pavlovic wrote:

George,

(with reduction of load to normal levels),

Probably it's not the best idea just to run samples, they are called 
samples for the reason  ;)


Basically you can run same Rally task two times before/after upgrade 
and compare results (Rally has sort of trends support)

Usually what I have heard from Ops guys is next:

  * Run Rally on periodic basis
  * Convert data Rally DB -> ElasticSearch
  * Build Kibana/Grafana on top


FYI I am working on making above scenario work out of the box.

--

So can you provide some more details on what you are looking for? 
What's missing?



Best regards,
Boris Pavlovic

On Tue, Aug 29, 2017 at 2:24 AM, George Shuklin 
> wrote:


Hello everyone.

Does someone do regression testing before performing upgrade
(within same major version)? How do you do this? Do you know any
tools for such tests? I started to research this area, and I see
three openstack-specific tools: rally (with reduction of load to
normal levels), tempest (can it be used by operators?) and granade.

If you use any of tools, how do you feel about them? Are they
worth time spent?



___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] regression testing before upgrade (operators view)

[Boris Pavlovic]

George,

(with reduction of load to normal levels),

Probably it's not the best idea just to run samples, they are called
samples for the reason  ;)

Basically you can run same Rally task two times before/after upgrade and
compare results (Rally has sort of trends support)
Usually what I have heard from Ops guys is next:

   - Run Rally on periodic basis
   - Convert data Rally DB -> ElasticSearch
   - Build Kibana/Grafana on top


FYI I am working on making above scenario work out of the box.

-- 

So can you provide some more details on what you are looking for? What's
missing?


Best regards,
Boris Pavlovic

On Tue, Aug 29, 2017 at 2:24 AM, George Shuklin 
wrote:

> Hello everyone.
>
> Does someone do regression testing before performing upgrade (within same
> major version)? How do you do this? Do you know any tools for such tests? I
> started to research this area, and I see three openstack-specific tools:
> rally (with reduction of load to normal levels), tempest (can it be used by
> operators?) and granade.
>
> If you use any of tools, how do you feel about them? Are they worth time
> spent?
>
>
> Thanks!
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] regression testing before upgrade (operators view)

[George Shuklin]

Hello everyone.

Does someone do regression testing before performing upgrade (within 
same major version)? How do you do this? Do you know any tools for such 
tests? I started to research this area, and I see three 
openstack-specific tools: rally (with reduction of load to normal 
levels), tempest (can it be used by operators?) and granade.


If you use any of tools, how do you feel about them? Are they worth time 
spent?



Thanks!


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators