If your external system is called synchronously by your tasks, you could
try setting the queue's max_concurrent_requests parameter in queue.yaml/xml
This is documented for Java, but the same general rules apply for Python or
Go applications.
http://code.google.com/appengine/docs/java/config/queue.html#Defining_Push_Queues_and_Processing_Rates
Nick Verne
On Mon, Oct 31, 2011 at 3:13 PM, Pol wrote:
> Hi,
>
> Through a task queue, we manage calls from GAE to an external system,
> which is fully-scalable but takes minutes to do so. The task queue
> rate is therefore set to 200/s.
>
> So what happens is that sometimes we get bursts of activity, and the
> task queue sends 200 requests at once: the vast majority fails in the
> external system as it can't handle this sudden request in load. But
> the external system doesn't start scaling up as there are no more
> requests coming and it's scaling based on the CPU load. Then suddenly
> you get another burst of requests from the task queue retrying, and
> then again the majority fails and so on.
>
> So how can we configure a task queue to have a very high rate *but*
> prevent it to get to this rate too fast? I noticed the bucket size
> parameter, but I'm not sure how to use it properly.
>
> Thanks,
>
> - Pol
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
