Simple task dispatching (How heavy is celery + RabbitMQ)

Hi, I'd like to use a light weight dispatching system for a web server. Basically some django post requests might require processing, which should be done in the back ground due to its run times. The results would be added to the django data base. The browser could verify via AJAX requests whet

Re: Simple task dispatching (How heavy is celery + RabbitMQ)

On Mon, Nov 28, 2011 at 11:14 AM, Gelonida N wrote: > Hi, > > I'd like to use a light weight dispatching system for a web server. > > Basically some django post requests might require processing, which > should be done in the back ground due to its run times. > > The results would be added to the

Re: Simple task dispatching (How heavy is celery + RabbitMQ)

You don't need to roll your own... have a look at https://github.com/dmgctrl/django-ztask It's based on ZeroMQ and really lightweight compared to any alternative with a broker (yes, it's brokerless but that's a good thing I think for most use cases which don't really need a broker). -- You re

Re: Simple task dispatching (How heavy is celery + RabbitMQ)

Celery has a bunch of alternative brokers, depending on your constrains you may not need something as "heavy" as RabbitMQ, you could for example use db broker if you don't have many tasks, it is slower but does not add a new server (apart from celery) to your stack, other option is to use Redis as

Re: Simple task dispatching (How heavy is celery + RabbitMQ)

I'm also using redis for both brokering and caching. So far, so good. My use case consists in one HUGE (5 hours) task, and several tiny (30s or less) scheduled tasks. Cheers, AT On Mon, Nov 28, 2011 at 10:54 AM, Carlos Daniel Ruvalcaba Valenzuela < clsdan...@gmail.com> wrote: > Celery has a bun

Re: Simple task dispatching (How heavy is celery + RabbitMQ)

Il giorno 28/nov/2011, alle ore 12:14, Gelonida N ha scritto: > Hi, > > I'd like to use a light weight dispatching system for a web server. > > Basically some django post requests might require processing, which > should be done in the back ground due to its run times. > > The results would be

Re: Simple task dispatching (How heavy is celery + RabbitMQ)

On 11/28/2011 12:14 PM, Gelonida N wrote: > Hi, > > I'd like to use a light weight dispatching system for a web server. > > Basically some django post requests might require processing, which > should be done in the back ground due to its run times. > > The results would be added to the django d

Re: Simple task dispatching (How heavy is celery + RabbitMQ)

There are plenty of tutorials about memory management with RabbitMQ. There are mechanisms for setting the high watermark for memory to throttle producers. You could look at the database backend for django-kombu. It might have a smaller footprint. django-ztask looks interesting, but you don'