[google-appengine] Re: Long running tasks impact on scheduler ?

2011-09-13 Thread stevep
Thanks for the explanation Jon (sorry I has used John before). Hopefully you all can continue to explore how TQ tasks can be managed separately by The Scheduler** for better instance optimization. cheers, stevep **Caps pun intended: http://www.imdb.com/title/tt0113762/ On Sep 12, 10:32 am, Jon

Re: [google-appengine] Re: Long running tasks impact on scheduler ?

2011-09-12 Thread Jon McAlister
Backends are one good way to do this. You can direct tasks at a backend, and then control the number of instances for that backend directly. http://code.google.com/appengine/docs/python/backends/overview.html Once the billing rollout is complete, the treatment of tasks and non-task requests,

Re: [google-appengine] Re: Long running tasks impact on scheduler ?

2011-09-12 Thread John
What about using the task header X-AppEngine-FailFast ? I understand that the task will be retried if no instance is available to serve it immediately - and no new instance will be spun up. Would that be a good idea ? We'd like to avoid using backend instances for tasks (to avoid paying

[google-appengine] Re: Long running tasks impact on scheduler ?

2011-09-10 Thread Gerald Tan
My guess is it will depend on whether the scheduler treats task requests the same as requests from outside. If they are treated similarly, then the scheduler would spin up new instances in an attempt to reduce the latency of the tasks... which would be silly. -- You received this message

[google-appengine] Re: Long running tasks impact on scheduler ?

2011-09-10 Thread stevep
+1 However please include sub-second tasks Just today I was looking at my logs/appstats. A client new recod write function I have that consists of three separate new kinds being put. It seems to run consistently at 250-300ms per HR put(). These occur serially: first one in my on-line handler,

Re: [google-appengine] Re: Long running tasks impact on scheduler ?

2011-09-10 Thread Robert Kluin
I'd very much like to know how long-running (over 1000ms) requests are treated by the new scheduler as well. Previously I believe they were basically ignored, and hence would not cause new instances to be spun up. And, yes I would very much like to have control over how task queues are treated