Yeah I've seen the djcelery solution before... I've tried implementing
it, but I'm getting an error on the celeryd console:
TypeError: processXLS1() got an unexpected keyword argument 'task_is_eager'
when I try running processXLS1.delay(dataObjectID=someInteger) from a
function in views.py
here's what's in my tasks.py:
"
from celery.decorators import task
from enzymeFilter.models import *
from django.db import transaction
@task
def printGene(y):
print "ppppppppppppppppppp"
fil=open('/var/www/media/testFile','w')
fil.write('coming from background')
fil.close()
print Gene.objects.get(id=y+1)
return True
@task
@transaction.commit_manually
def processXLS1(datasetObjectID):
print "processing XLS as task"
datasetObject = Dataset.objects.get(id=datasetObjectID)
try:
... more processing code
"
thanks
-Nathan
On Sun, Nov 27, 2011 at 9:52 PM, Brian Schott <bfsch...@gmail.com> wrote:
> You really should look at django-celery and RabbitMQ. The upload submit can
> initiate a task that is defined in tasks.py. There are separate worker
> processes that pick up the task from the message queue and do the actual
> work. These workers don't even have to be on the same machine with rabbitMQ
> so you get instant scalability. Then your AJAX job status view can poll a
> job status table that is updated by the task and you don't have to worry
> about threads.
> https://github.com/ask/django-celery
>
> Brian Schott
> bfsch...@gmail.com
>
>
>
> On Nov 27, 2011, at 8:54 PM, Nathan McCorkle wrote:
>
>> I'm using Django 1.3 and am processing 3 files into genes, proteins,
>> and genomes for a tool I built... this processing usually takes
>> between 5 minutes to a few hours, depending on the genome size.
>>
>> After uploading the files (10-100 MB), the upload view begins
>> processing the files, without returning for a long time (causing my
>> browser to ask me if I want to kill the plupload script).
>>
>> For the user of this app, they don't know if their upload failed or is
>> processing, etc... so I'd think forking the processing function (or
>> making it a new thread) is what needs to happen, then the view can
>> return to the user, and the upload page can start doing AJAX requests
>> to check to processing progress.
>>
>> I feel like I want to do this:
>> http://www.artfulcode.net/articles/threading-django/
>> but am scared of crashing because I've heard Django isn't threadsafe
>> with transactions. (and my processing function is making DB
>> transactions)
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To post to this group, send email to django-users@googlegroups.com.
>> To unsubscribe from this group, send email to
>> django-users+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/django-users?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.
>
>
--
Nathan McCorkle
Rochester Institute of Technology
College of Science, Biotechnology/Bioinformatics
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
