On Wed, Jan 28, 2015 at 11:39 AM, Flavio Percoco <fla...@redhat.com> wrote:
> On 28/01/15 10:23 +0200, Denis Makogon wrote: > >> >> >> On Tue, Jan 27, 2015 at 10:26 PM, Gordon Sim <g...@redhat.com> wrote: >> >> On 01/27/2015 06:31 PM, Doug Hellmann wrote: >> On Tue, Jan 27, 2015, at 12:28 PM, Denis Makogon wrote: >> I'd like to build tool that would be able to profile >> messaging over >> various deployments. This "tool" would give me an ability to >> compare >> results of performance testing produced by native tools and >> oslo.messaging-based tool, eventually it would lead us into >> digging >> into >> code and trying to figure out where "bad things" are happening >> (that's >> the >> actual place where we would need to profile messaging code). >> Correct me >> if >> i'm wrong. >> >> >> It would be interesting to have recommendations for deployment of >> rabbit >> or qpid based on performance testing with oslo.messaging. It would >> also >> be interesting to have recommendations for changes to the >> implementation >> of oslo.messaging based on performance testing. I'm not sure you >> want >> to >> do full-stack testing for the latter, though. >> >> Either way, I think you would be able to start the testing without >> any >> changes in oslo.messaging. >> >> I agree. I think the first step is to define what to measure and then >> construct an application using olso.messaging that allows the data of >> interest to be captured using different drivers and indeed different >> configurations of a given driver. >> >> I wrote a very simple test application to test one aspect that I felt >> was >> important, namely the scalability of the RPC mechanism as you increase >> the >> number of clients and servers involved. The code I used is https:// >> github.com/grs/ombt, its probably stale at the moment, I only link to >> it as >> an example of approach. >> >> Using that test code I was then able to compare performance in this one >> aspect across drivers (the 'rabbit', 'qpid' and new amqp 1.0 based >> drivers >> _ I wanted to try zmq, but couldn't figure out how to get it working >> at the >> time), and for different deployment options using a given driver (amqp >> 1.0 >> using qpidd or qpid dispatch router in either standalone or with >> multiple >> connected routers). >> >> There are of course several other aspects that I think would be >> important >> to explore: notifications, more specific variations in the RPC >> 'topology' >> i.e. number of clients on given server number of servers in single >> group >> etc, and a better tool (or set of tools) would allow all of these to be >> explored. >> >> From my experimentation, I believe the biggest differences in >> scalability >> are going to come not from optimising the code in oslo.messaging so >> much as >> choosing different patterns for communication. Those choices may be >> constrained by other aspects as well of course, notably approach to >> reliability. >> >> >> >> >> After couple internal discussions and hours of investigations, i think >> i've >> foung the most applicabale solution >> that will accomplish performance testing approach and will eventually be >> evaluated as messaging drivers >> configuration and AMQP service deployment recommendataion. >> >> Solution that i've been talking about is already pretty well-known across >> OpenStack components - Rally and its scenarios. >> Why it would be the best option? Rally scenarios would not touch messaging >> core part. Scenarios are gate-able. >> Even if we're talking about internal testing, scenarios are very useful >> in this >> case, >> since they are something that can be tuned/configured taking into account >> environment needs. >> >> Doug, Gordon, what do you think about bringing scenarios into messaging? >> > > I personally wouldn't mind having them but I'd like us to first > discuss what kind of scenarios we want to test. > > I'm assuming these scenarios would be pure oslo.messaging scenarios > and they won't require any of the openstack services. Therefore, I > guess these scenarios would test things like performance with many consumers, performance with several (a)synchronous calls, etc. What > performance means in this context will have to be discussed as well. > Correct, oslo.messaging scenarios would expect to have only AMQP service and nothing else. Yes, that's what i've been thinking about. Also, i'd like to share doc that i've found, see [1]. As i can see it would be more than useful to enable next scenarios: - Single multi-thread publisher (rpc client) against single multi-thread consumer - using RPC cast/call methods try to measure time between request and response. - Multiple multi-thread publishers against single multi-thread consumer - using RPC cast/call methods try to measure time between requests and responses to multiple publishers. - Multiple multi-thread publishers against multiple multi-thread consumers [1] http://agocs.org/blog/2014/08/19/rabbitmq-best-practices-in-go/ > In addition to the above, it'd be really interesting if we could have > tests for things like reconnects delays, which I think is doable with > Rally. Am I right? > > That's might be the case. But i'd ask Rally team first. > Cheers, > Flavio > > > >> >> ____________________________________________________________ >> ______________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject: >> unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> Kind regards, >> Denis M. >> >> > ____________________________________________________________ >> ______________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject: >> unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > -- > @flaper87 > Flavio Percoco > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > Kind regards, Denis M.
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev