Re: Hand off time-consuming task to other thread/process/application/coroutine...?

2013-01-30 Thread Arndt Droullier
2013/1/29 Maarten De Schrijver > @Arndt, > > I can run multiple instances of my application but how exactly can I then > "route the slow running tasks to the one and the rest to the other"? > Could you also elaborate on the session cache? > > > Hi, for example, I use nginx to proxy requests to my

Re: Hand off time-consuming task to other thread/process/application/coroutine...?

2013-01-29 Thread Jonathan Vanasco
On Jan 29, 3:26 am, Maarten De Schrijver wrote: > Now, in my idea: wouldn't it be better, in the second step (page 2) to > just immediately persist the user submitted data to the SQLite database to > allow the user to continue the flow and to hand off the connection to the > remote CRM to anoth

Re: Hand off time-consuming task to other thread/process/application/coroutine...?

2013-01-29 Thread Maarten De Schrijver
@Arndt, I can run multiple instances of my application but how exactly can I then "route the slow running tasks to the one and the rest to the other"? Could you also elaborate on the session cache? @Jonathan, the response or fetched information returning from the "slow running task" (the conn

Re: Hand off time-consuming task to other thread/process/application/coroutine...?

2013-01-28 Thread Maarten De Schrijver
Celery seems interesting. Trying to get it to work, but getting some errors. I'll keep you posted. Maarten On Monday, 28 January 2013 12:53:58 UTC+1, Jesaja Everling wrote: > > Hi Maarten, > > I would suggest looking into using a task queue like Celery. > http://celeryproject.org/ > > There a

Re: Hand off time-consuming task to other thread/process/application/coroutine...?

2013-01-28 Thread Jonathan Vanasco
I don't think you'd necessarily need to use celery/redis/etc for this. No matter what you use to fetch the remote system data , you'll need to implement a user pattern like this: 1. User loads the web page , which is 'blank' with a loading message 2. The web-page fires a javascript call to an API

Re: Hand off time-consuming task to other thread/process/application/coroutine...?

2013-01-28 Thread Arndt Droullier
Have you considered to run two instances of your pyramid application and simply route the slow running tasks to the one and the rest to the other? There would be - no need need to rewrite code, - no danger the main applications blocks and - for the slow running app webservers have reliable connecti

Re: Hand off time-consuming task to other thread/process/application/coroutine...?

2013-01-28 Thread Whit Morriss
Couple other resque clones worth mention: pyres and my personal favorite retools.queue: http://retools.readthedocs.org/en/latest/api/queue.html#module-retools.queue -whit On Jan 28, 2013, at 5:53 AM, Jesaja Everling wrote: > Hi Maarten, > > I would suggest looking into using a task queue l

Re: Hand off time-consuming task to other thread/process/application/coroutine...?

2013-01-28 Thread Jesaja Everling
Hi Maarten, I would suggest looking into using a task queue like Celery. http://celeryproject.org/ There are also a few alternatives like http://kr.github.com/beanstalkd/ and http://nvie.com/posts/introducing-rq/, your you could roll your own solution using Redis. But Celery works really well for

Hand off time-consuming task to other thread/process/application/coroutine...?

2013-01-28 Thread Maarten De Schrijver
Hello, I've written an application in Pyramid. In one particular view, I need to connect to a kind of CRM system via SOAP-XML (need to check if the user exists and if not add to this CRM system). It works fine, but, it is slow: the users web browser "hangs" (or rather: is busy) sometimes for ov