On Oct 21, 2009, at 11:55 AM, Daniel Roseman wrote:
> > On Oct 21, 9:28 am, Mike Thon <mike.t...@gmail.com> wrote: >> I'm new to web programming and I have a basic question about the >> design of my Django application. my application will do some number >> crunching on data files uploaded by users. The data processing will >> take from minutes to hours for each job. I don't expect to ever >> get a >> large number of concurrent users but I'd still like to set it up so >> that I can control the maximum number of data processing jobs that >> are >> run in parallel. I was planning to write a simple FIFO queue manager >> (in fact I think there is a python package for this) and then run the >> data processing in separate threads. I'm also planning to use the >> Django data model for storing the data so I would have multiple >> threads writing to the data store. What is not clear to me is what >> happens when I have more than one visitor to the site. Are multiple >> instances of my Django app launched, one per visitor? I need to >> ensure that I only have one queue manager running on the server, not >> one per visitor. I would be using Apache and either mySQL or sqlite3 >> as the database, in case that matters. >> >> thanks for any help >> Mike > > Take a look at the Celery project[1]. This is a great distributed task > queue for Django that I think will do exactly what you need - each job > request is sent to the queue and managed there, so you don't need to > worry about multiple instances. > > [1]:http://ask.github.com/celery/introduction.html > Thanks for pointing me towards celery. Its probably overkill for what I want to do right now but I'm going to try to set it up anyway. Mike --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
