Hi there,
Last night I experimented with task queues to see what level of
concurrency I could achieve when running on the live environment.
Summary of the test app:
- Bulk load 30,000 entities of a given type (3 properties / entity
object).
- Command line job I ran from my PC that hit an URL to queue the
entries
- This program was multi-threaded so I could simulate a bit of
load (10 concurrent threads)
- Queueing URL created a task queue entry within the same app
- 2nd URL handled the task queue request and stored entity to the
Datastore
I watched the task queue dashboard for a few minutes and observed a
few things:
- Enqueue rate quickly outpaced dequeue rate
- I was enqueing at about 12 requests / second, but dequeuing at
4 requests / second
- GAE did not appear to increase the dequeue rate over time in
response to my queue depth
Result: It took a very long time to dequeue 30,000 tasks (over 2
hours). It seemed that GAE was running one instance of my app.
Expected: Much higher throughput.
Is this expected behavior? It seems that given the 30 second request
limit that task queues are an important way to increase throughput
(ala MapReduce). But the "swarm" of app instances never seemed to
arrive.
thanks
-- James
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en
-~----------~----~----~----~------~----~------~--~---
