On Oct 13, 2010, at 5:07 PM, ringemup wrote: > Thank you for taking the time to explain that, Shawn -- it makes > everything a LOT clearer. > > If you could spare the time, I'm curious about a couple of aspects of > the architecture: > > - What is the purpose of having a separate broker and daemon? > - How does the broker know when to pass the task back to Celery? > - Is there a reason other than resource usage for the broker to live > on a different machine? > > Also, can this all be run in a shared hosting environment, or are root > permissions needed to install Celery and RabbitMQ? > > >
The whole "messaging" system as implemented in RabbitMQ is very complex (and beyond my understanding). So, I'll just say that the broker and daemon are separate because it's a lot more flexible this way -- RabbitMQ is used for things you and I haven't even thought of. So although a competing software product that did the simplified process flow I described could work for asynchronous processing in a Django app, but wouldn't be viable for many other uses. Because of open source, we have the advantage of an enterprise-level piece of software for free, and we can choose to use it even for trivial needs. The broker has buckets known as 'vhosts.' The default configuration suggested by the django-celery docs just has you configure one vhost. The Celery daemon is configured to look for a specific vhost. So that's how the broker knows when to pass specific tasks to the daemon. You could have many vhosts on one broker. In fact, you could have any number of completely unrelated applications using the same broker with separate vhosts. The broker could live on the same machine. I'd say that, in addition to resource usage, you'd want the broker on a separate server because any number of applications on any number of servers might rely on it. So if a server died, or had to be taken down for maintenance, your broker would still be available to others. In addition, if other servers were putting tasks in your broker that your server was expected to execute, it could then retrieve and execute them when it came back up -- no loss of messages. Shawn -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
