On Wed, Jun 03, 2009 at 10:43:34AM -0700, Minesh Patel wrote: > For this, since your data is huge and seems like it will need a lot of > CPU utilization, you might have to deferToThread, since Twisted runs > in a single thread and will block AFAIK. > > d = threads.deferToThread(do_a, arg1_to_do_a, arg2...) > d.addCallback(do_a2, ...) >
I delayed and forgot to respond to this thread. Sorry for that. Actually, I had asked the question on doing asynchronous monitoring of new file arrival and then based on the file contents, doing tasks a,b,c asynchronously. Thank you for your responses. I got a number of pointers to look at. * As a newbie to this world, I just started studying further. I realized that in my application, I have the control form the client on when I can notify that a particular file can be looked at. So, for the time being, I just skipped the inotify linux system related calls ( BTW, Twsited + inotify works great) and when ahead with just implementing a Twisted a Server which listens to a particular port and when receives data, will either read the file or parse the data to do some activity (like fetch logs from database). threads.deferToThread(func,[args]) is what I am using. http://paste.pocoo.org/show/122155/ I am still working out more details as doing this threads.deferToThread calls effectively, because there could be number of clients parallely requesting it and the data from my query (internal function) could be so big that it might take 30 minutes to service a request. * Another thing is when using something like "from datasources import query" I could not make it to application (.tac to run it from twistd), it could nto find datasources in its PYTHONPATH. Got to figure out how to do it, which I might pretty soon. Thanks again, Senthil _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
