Hi Bill, I think what you described is possible with Supervisor. We're using ActiveMQ with hundreds of worker processes spread over several machines. For what it's worth, I have some doubt about automatically rebalancing workers based on load. In my experience, heavy queuing is often the result of either slow external resources (DB locks, slow web service calls) or overall machine resource issues like CPU load. In a lot of these cases, adding more workers on a queue does not help b/c the underlying issue remains. In the case of high CPU load, adding more of the CPU-intense workers can exacerbate the problem.
Just my 2 cents. Cheers, Roger On Wed, Feb 2, 2011 at 5:46 AM, Bill Moseley <[email protected]> wrote: > Anyone here using using Supervisor to manage RabbitMQ consumers? Could you > describe your configuration? > > My goal is to have centralized monitoring and control of worker processes > (RabbitMQ consumers) over a collection of machines. I want to integrate > with the rabbitmqctl tool that lists queues and their sizes so that I can > rebalance the worker processes based on changing load. > > For a simple example, I might notice that the queue creating reports is > getting large and that the queue for rotating photos is empty, so I might > want to stop half the image rotation processes and double the number of > report generation processes. > > I understand Supervisor doesn't have a direct way to interface and display > output from rabbitmqctl, but I could use the XML-RPC API to tie rabbitmqclt > and Supervisor together, for example. > > I'm more curious if what I describe above is a good use for Supervisor, and > if anyone is using it in a similar way. > > Thanks, > > > > -- > Bill Moseley > [email protected] > > _______________________________________________ > Supervisor-users mailing list > [email protected] > http://lists.supervisord.org/mailman/listinfo/supervisor-users > >
_______________________________________________ Supervisor-users mailing list [email protected] http://lists.supervisord.org/mailman/listinfo/supervisor-users
