[google-appengine] Re: again "Request was aborted after waiting too long..."

2010-08-27 Thread David Burns
My site is constantly serving so should always been "warm" according to the Dashboard. My AppStats don't show any particular spikes or any areas that are slow. When the datastore was serving really slowly I did a number of optimizations to make sure that DataStore access was minimal to a point I c

[google-appengine] Re: again "Request was aborted after waiting too long..."

2010-08-26 Thread Jan Z/ Hapara
Just as an experiment we ran this job again, this time using JMeter to "prime" the VM's at a sustained 25 requests per second for 15-20 mins before the tasks were queued up. No impact - same request aborted rate as before. So it doesn't seem like it's a problem with availability of "warm" vm's...

[google-appengine] Re: again "Request was aborted after waiting too long..."

2010-08-26 Thread Jan Z/ Hapara
Had another data run complete just now - a url-fetch focused job (talking to Google services only, no rpc, no db, little memcache) with 2,500 tasks spread across 5 queues at 6/m each. Average task is 4-8 urlfetches, runs in 17s, with < 100msec CPU time. There was minimal to no other load on the

[google-appengine] Re: again "Request was aborted after waiting too long..."

2010-08-26 Thread Jan Z/ Hapara
Geoffrey, can you elaborate on "the criteria for creating a new instance" please? Clearly there are some thresholds being calculated and throttles being applied here, but it's wholly unclear what they are or what design patterns need to be applied to avoid them. There was some guidance from the G

[google-appengine] Re: again "Request was aborted after waiting too long..."

2010-08-26 Thread Geoffrey Spear
On Aug 26, 8:39 am, David Burns wrote: > I am getting this error too and I am not doing any URL fetches. I have > put a lot of logging throughout my application and getting this error > before any of my logging is hit > > # > >    1. >       08-25 05:30AM 43.978 / 500 10011ms 0cpu_ms 0kb Mozilla

[google-appengine] Re: again "Request was aborted after waiting too long..."

2010-08-26 Thread David Burns
I am getting this error too and I am not doing any URL fetches. I have put a lot of logging throughout my application and getting this error before any of my logging is hit # 1. 08-25 05:30AM 43.978 / 500 10011ms 0cpu_ms 0kb Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534

[google-appengine] Re: again "Request was aborted after waiting too long..."

2010-08-26 Thread Jan Z/ Hapara
A mix. We have a work package that is pretty exclusively async and another just using gdata (with zero async). There is no discernible difference, performance-wise. J On Aug 26, 10:46 pm, Tim Hoffman wrote: > Are you doing using async urlfetches ? > > T > > On Aug 26, 6:03 pm, "Jan Z/ Hapara"

[google-appengine] Re: again "Request was aborted after waiting too long..."

2010-08-26 Thread Tim Hoffman
Are you doing using async urlfetches ? T On Aug 26, 6:03 pm, "Jan Z/ Hapara" wrote: > Tim, your app is probably not doing much urlfetching? > > We have an app (h-script) that performs several urlfetch operations > per task.  We are talking to other Google services only.  On a good > day, each ta

[google-appengine] Re: again "Request was aborted after waiting too long..."

2010-08-26 Thread Jan Z/ Hapara
Tim, your app is probably not doing much urlfetching? We have an app (h-script) that performs several urlfetch operations per task. We are talking to other Google services only. On a good day, each task takes between 3 and 10 seconds. This goes up to 20 seconds sometimes. The CPU time for the

[google-appengine] Re: again "Request was aborted after waiting too long..."

2010-08-26 Thread Tim Hoffman
Unfortunately you haven't provided a great deal of information about what you are doing in your tasks. My guess is whatever you are running in the task is just taking too long. Queued will retry unless you exit cleanly. And each failed tasks time between retries will will increase. You should p

[google-appengine] Re: again "Request was aborted after waiting too long..."

2010-08-25 Thread Jan Z/ Hapara
Not relevant unfortunately. There is not try / catch here - when this happens, your code doesn't even start executing - GAE just fails it outright, and worse yet, the back-off algorithm kicks in so after a few generations the whole thing degenerates into uselessness (I've seen tasks that fail with