On 10/06/14 15:03 +0400, Dina Belova wrote:
Hello, stackers!Oslo.messaging is future of how different OpenStack components communicate with each other, and really I’d love to start discussion about how we can make this library even better then it’s now and how can we refactor it make more production-ready. As we all remember, oslo.messaging was initially inspired to be created as a logical continuation of nova.rpc - as a separated library, with lots of transports supported, etc. That’s why oslo.messaging inherited not only advantages of now did the nova.rpc work (and it were lots of them), but also some architectural decisions that currently sometimes lead to the performance issues (we met some of them while Ceilometer performance testing [1] during the Icehouse). For instance, simple testing messaging server (with connection pool and eventlet) can process 700 messages per second. The same functionality implemented using plain kombu (without connection pool and eventlet) driver is processing ten times more - 7000-8000 messages per second. So we have the following suggestions about how we may make this process better and quicker (and really I’d love to collect your feedback, folks): 1) Currently we have main loop running in the Executor class, and I guess it’ll be much better to move it to the Server class, as it’ll make relationship between the classes easier and will leave Executor only one task - process the message and that’s it (in blocking or eventlet mode). Moreover, this will make further refactoring much easier.
To some extent, the executors are part of the server class since the later is the one actually controlling them. If I understood your proposal, the server class would implement the event loop, which means we would have an EventletServer / BlockingServer, right? If what I said is what you meant, then I disagree. Executors keep the eventloop isolated from other parts of the library and this is really important for us. One of the reason is to easily support multiple python versions - by having different event loops. Is my assumption correct? Could you elaborate more?
2) Some of the drivers implementations (such as impl_rabbit and impl_qpid, for instance) are full of useless separated classes that in reality might be included to other ones. There are already some changes making the whole structure easier [2], and after the 1st issue will be solved Dispatcher and Listener also will be able to be refactored.
This was done on purpose. The idea was to focus on backwards compatibility rather than cleaning up/improving the drivers. That said, sounds like those drivers could user some clean up. However, I think we should first extend the test suite a bit more before hacking the existing drivers.
3) If we’ll separate RPC functionality and messaging functionality it’ll make code base clean and easily reused.
What do you mean with this?
4) Connection pool can be refactored to implement more efficient connection reusage.
Please, elaborate. What changes do you envision? As Dims suggested, I think filing some specs for this (and keeping the proposals separate) would help a lot in understanding what the exact plan is. Glad to know you're looking forward to help improving oslo.messaging. Thanks, Flavio
Folks, are you ok with such a plan? Alexey Kornienko already started some of this work [2], but really we want to be sure that we chose the correct vector of development here. Thanks! [1] https://docs.google.com/document/d/ 1ARpKiYW2WN94JloG0prNcLjMeom-ySVhe8fvjXG_uRU/edit?usp=sharing [2] https://review.openstack.org/#/q/ status:open+owner:akornienko+project:openstack/oslo.messaging,n,z Best regards, Dina Belova Software Engineer Mirantis Inc.
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-- @flaper87 Flavio Percoco
pgpTe9lXrjjsY.pgp
Description: PGP signature
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev