On Fri, Oct 18, 2013 at 10:17 AM, Boris Pavlovic <[email protected]> wrote:
> John, > > Actually seems like a pretty good suggestion IMO, at least something worth > some investigation and consideration before quickly discounting it. Rather > than "that's not what tempest is", maybe it's something tempest "could do". > Don't know, not saying one way or the other, just wondering if it's worth > some investigation or thought. > > > These investigations I made before start working around Rally. It was > about 3 months ago. > It is not "quickly discounting" I didn't have yesterday time to make long > response, so I will write it today: > > I really don't like to make a copy of another projects, so I tried to > reuse all projects & libs that we already have. > > To explain why we shouldn't merge Rally & Tempest in one project (and > should keep both) we should analyze their use cases. > > > 1. DevStack - one "click" and get your OpenStack cloud from sources > > 2. Tempest - one "click" and get your OpenStack Cloud verified > > Both of these projects are great, because they are very useful and solve > complicated tasks without "pain" for end user. (and I like them) > > 3. Rally is also one "click" system that solve OpenStack benchmarking. > > To clarify situation. We should analyze what I mean by one "click" > benchmarking and what are the use cases. > > Use Cases: > 1. Investigate how deployments influence on OS performance (find the set > of good OpenStack deployment architectures) > 2. Automatically get numbers & profiling info about how your changes > influence on OS performance > 3. Using Rally profiler detect scale & performance issues. > Like here when we are trying to delete 3 VMs by one request they are > deleted one by one because of DB lock on quotas table > http://37.58.79.43:8080/traces/0011f252c9d98e31 > 4. Determine maximal load that could handle production cloud > > To cover these cases we should actually test OpenStack deployments making > simultaneously OpenStack API calls. > > So to get results we have to: > 1. Deploy OpenStack cloud somewhere. (Or get existing cloud) > 2. Verify It > 3. Run Benchmarks > 4. Collect all results & present it in human readable form. > > > The goal of Rally was designed to automate these steps: > 1.a Use existing cloud. > 1.b.1 Automatically get (virtual) Servers from (soft layer, Amazon, > RackSpace or you private public cloud, or OpenStack cloud) > 1.b.2 DeployOpenStack on these servers from source (using Devstack, Anvli, > Fuel or TrippleO...). > 1.b.3 Patch this OpenStack with tomograph to get profiling information (I > hope we will merge these patches into upstream). > 2. Using tempest verify this cloud (we are going to switch from > fuel-ostf-tests) > 3. Run specified parametrized (to be able to make different load) > benchmark scenarios > 4. Collect all information about execution & present it in human readable > form. (Tomograph, Zipking, matplotlib...) > > > So I am not sure that we should put inside Tempest Rally, because Rally > use tempest. It is something like putting Nova into Cinder =) > Also putting Tempest into Rally is not a good idea. (same as putting > Cinder back to Nova). > > > Best regards, > Boris Pavlovic > --- > Mirantis Inc. > > > On Thu, Oct 17, 2013 at 11:56 PM, John Griffith < > [email protected]> wrote: > >> >> >> >> On Thu, Oct 17, 2013 at 1:44 PM, Jay Pipes <[email protected]> wrote: >> >>> On 10/17/2013 03:32 PM, Boris Pavlovic wrote: >>> >>>> Jay, >>>> >>>> >>>> Or, alternately, just have Rally as part of Tempest. >>>> >>>> >>>> Actually, tempest is used only to verify that cloud works properly. >>>> And verification is only small part of the Rally. >>>> >>>> At this moment we are using fuel-ostf-tests, but we are going to use >>>> tempest to verify cloud. >>>> >>> >>> OK, cool... was just a suggestion :) Tempest has a set of stress tests >>> [1] which are kind of related, which is the only reason I brought it up. >>> >>> Best, >>> -jay >>> >>> [1] >>> https://github.com/openstack/**tempest/tree/master/tempest/**stress<https://github.com/openstack/tempest/tree/master/tempest/stress> >>> >>> >>> ______________________________**_________________ >>> OpenStack-dev mailing list >>> [email protected].**org <[email protected]> >>> http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-dev<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev> >>> >> >> Actually seems like a pretty good suggestion IMO, at least something >> worth some investigation and consideration before quickly discounting it. >> Rather than "that's not what tempest is", maybe it's something tempest >> "could do". Don't know, not saying one way or the other, just wondering if >> it's worth some investigation or thought. >> >> By the way, VERY COOL!! >> >> >> _______________________________________________ >> OpenStack-dev mailing list >> [email protected] >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > Boris, Great, thanks for the further explanation. I was in no way saying that *you* discounted it quickly, but at first glance I personally thought it was worth looking at. I mostly wanted to point out the consideration here and your detailed response here pretty much answers the questions that I had. I appreciate the details. John
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
