I'm currently using the pull queue with tags, and have experienced 
TranscientError in the past.
Thanks to support cases still being there I can go back in time with 
precise figures :) 

Here are my usage figures if it can help:

Currently:

Daily average 7-8 tasks/second enqueued over 2 tags, with a spike at 37 
tasks/second over 5 minutes.
One worker per tag that wakes up every 30 minutes to lease them.
No transcient error.


Oct 2012 (they probably improved the service since):

We were using many different tags (basically regrouping events per user 
with several million users).
1,800 tasks/sec enqueued, can peak at 2,500. Up to 100 requests/seconds 
that will lease tasks from the same pull task queue - but on different tags.
=> lots of TranscientError and DeadlineExceededError.

To mitigate this the solution was to spread the tasks over multiple task 
queues:
20 pull task queues instead of one, each pull task queue had 35 tasks added 
by second, and 4 requests per second were leasing and deleting tasks from 
that same queue.
I did still have regularly DeadlineExceeeded errors mainly about the call 
to delete tasks. Retrying at once solves it most of the time. I still found 
a few TranscientError (mostly on tasks delete too) but in such rate that 
made them acceptable.




-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

Reply via email to