Pasting a few things from IRC here to fill out the context... <marun> carl_baldwin: but according to markmcclain and salv-orlando, it isn't possible to trivially use multiple workers for rpc because processing rpc requests out of sequence can be dangerous
<carl_baldwin> marun: I think it is already possible to run more than one RPC message processor. If the neutron server process is run on multiple hosts in active/active I think you end up getting multiple independent RPC processing threads unless I'm missing something. <marun> carl_baldwin: is active/active an option? I checked one of my environments where there are two API servers running. It is clear from the logs that both servers are consuming and processing RPC messages independently. I have not identified any problems resulting from doing this yet. I've been running this way for months. There could be something lurking in there preparing to cause a problem. I'm suddenly keenly interested in understanding the problems with processing RPC messages out of order. I tried reading the IRC backlog for information about this but it was not clear to me. Mark or Salvatore, can you comment? Not only is RPC being handled by both physical servers in my environment but each of the API server worker processes is consuming and processing RPC messages independently. So, I am currently running a multi-process RPC scenario now. I did not intend for this to happen this way. My environment has something different than the current upstream. I confirmed that with current upstream code and the ML2 plugin only the parent process consumes RPC messages. It is probably because this environment is still using an older version of my multi-process API worker patch. Still looking in to it. Carl On Thu, Dec 5, 2013 at 7:32 AM, Carl Baldwin <c...@ecbaldwin.net> wrote: > Creating separate processes for API workers does allow a bit more room > for RPC message processing in the main process. If this isn't enough > and the main process is still bound on CPU and/or green > thread/sqlalchemy blocking then creating separate worker processes for > RPC processing may be the next logical step to scale. I'll give it > some thought today and possibly create a blueprint. > > Carl > > On Thu, Dec 5, 2013 at 7:13 AM, Maru Newby <ma...@redhat.com> wrote: >> >> On Dec 5, 2013, at 6:43 AM, Carl Baldwin <c...@ecbaldwin.net> wrote: >> >>> I have offered up https://review.openstack.org/#/c/60082/ as a >>> backport to Havana. Interest was expressed in the blueprint for doing >>> this even before this thread. If there is consensus for this as the >>> stop-gap then it is there for the merging. However, I do not want to >>> discourage discussion of other stop-gap solutions like what Maru >>> proposed in the original post. >>> >>> Carl >> >> Awesome. No worries, I'm still planning on submitting a patch to improve >> notification reliability. >> >> We seem to be cpu bound now in processing RPC messages. Do you think it >> would be reasonable to run multiple processes for RPC? >> >> >> m. >> >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev