That actually tells me why I experience some issues from time to time. This is how I have worked around these kind of "limit" feature (and note I have called it a "feature" and not an issue - coding for the cloud means living in a community where some entity will have to police use of resources and ensure apps are well behaved citizens).
(1) I use task queues and go parallel like there is no tomorrow. Everything I do (save for things that respond to the client side in a symmetric form) is asymmetric and uses task queues. I use the same task queue (really, no need to do more unless you need to worry about order of execution - which I don't - or "explicit" (not GAE imposed) re-execution of the same task (persistent tasks - multi-state tasks). Of course, when firing off rapidly to the same task queue (and the same handler - which is what I do), then you push against GAE rules re concurrent and/or request/second ceilings. So, what I do is I use the same servlet (oh, this is all Java, but the same rules apply to Python apps) but with with different endpoints (in my web.xml, I have multiple ones pointing to the same 'class' servlet). Then I use a random number generator to choose which endpoint will be used for a particular task queue operation [remember, unless you use the data store, piggyback data, or use memcache, you do not share data across servlets (and/or instances of the same servlet) - in fact, I code to put myself in situations (as I rely on the parallel nature of the engine) where I do not need to sync activities and use of same resources (in some cases, I have to, and I use memcache -- but I am learning not to sync at all and rely on best effort basis (almost heuristic in nature).] The end result is that I break down any activity to unrelated activities and I let them run in the background while I return to the client with some registration key for retrieving results. How does that work? Easy. It is almost like wave (to the user) but it is not wave - so I call it "wnw"). That's a totally different discussion. Anyway, what the user sees is quick feedback. What the server gets to do is jobs that run (if needed) for 2 years on GAE without running against any limit - well, make sure it is a PAID account). All things happen in parallel (so, GAE is in fact very fast for me - no complains there). I make sure I watch for datasource and other errors - and I do lots of retries and other things to make it fail-safe et cetera. (2) My units of work (tasks - endpoints that run tasks) can be run multiple times; they actually store their last known run state (consistently) encoded in the message data itself. If a task is not complete, it re-queues itself with the new state info; if it can't (because GAE kills it), it tries again. (3) I consider GAE issues as its features and its source of strength; I think alignment and I take advantage by coding for it; I have a hard real-time embedded (highly pervasive) background as well, and to me, GAE is a soft real-time OS. Code for it in that context. Nothing should take more than 3 seconds to run (in this case). If it does, break it down. Use persistent tasks - as I explained - to do long activities (will post something to my blog in a week or so once I am passed doing some demo projects). Take advantage of parallel nature of things; but that will affect how coordinate with the client side and increases communications overhead. Think of task queue as a big eventbus. That's what it is anyway. Code for that. If you consider these, then app engine is a unique system that gives you max throughout without you having to run servers. Now, that is really unique these days. Why stress test ... assume failure is part of the process. Code for it -- at client and server end. Try try try. There has never been a good solution (ever) for hiding idiosyncrasies of highly "geographically" distributed/redundant systems. GAE acts as if it does, but you get the unexpected failures. Code for it. Cheers! Bob/babakm, developer @ intillium.com On Sep 24, 6:07 am, "Nick Johnson (Google)" <nick.john...@google.com> wrote: > Hi David, > As long as you don't intend to exceed 30 concurrent requests (about 400qps > if your requests take 75 milliseconds each), there's no need to notify us. > > -Nick Johnson > > > > On Wed, Sep 23, 2009 at 4:53 PM, Dai bok <rankin...@gmail.com> wrote: > > > Hi all, > > > I am busy stress testing my application, and I just thought that it > > might be an idea to let the people at google know that I inented to > > test some sections of my application that may come under high load. > > > What is the policy for this? Do I need to let GAE admins know about > > this? Who should I contact? > > > Thanks for your time, > > David > > -- > Nick Johnson, Developer Programs Engineer, App Engine > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: > 368047 --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
