Re: [Openstack-operators] regression testing before upgrade (operators view)

2017-08-31 Thread George Shuklin
Hello. Thank you for reply. In those days I've tested tempest and rally. I found rally more suited for my goals. Insofar I'd like to admit that it's too overloaded and when I start poking around I quickly stuck with 'verify' mode, which in turn stuck in our neutron configuration (we have no

Re: [Openstack-operators] regression testing before upgrade (operators view)

2017-08-29 Thread Boris Pavlovic
George, (with reduction of load to normal levels), Probably it's not the best idea just to run samples, they are called samples for the reason ;) Basically you can run same Rally task two times before/after upgrade and compare results (Rally has sort of trends support) Usually what I have

[Openstack-operators] regression testing before upgrade (operators view)

2017-08-29 Thread George Shuklin
Hello everyone. Does someone do regression testing before performing upgrade (within same major version)? How do you do this? Do you know any tools for such tests? I started to research this area, and I see three openstack-specific tools: rally (with reduction of load to normal levels),