[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-10-27 Thread nickmilon
Appl id:geognos appl version : 5 language :python live instances at the time: 2 loading request: no A typical normal operation for this request :" 10-27 09:47AM 08.920 /api/countries/info/CL.json 200 17ms 19cpu_ms 0kb MSIE 15.0,gzip(gfe) 195.97.105.127 - - [27/Oct/2010:09:47:08 -0700] "GET /api/

Re: [google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-10-27 Thread Derrick Schneider
Yeah, I'm in the same boat. We're trying to decide which implementation of this app to go with, and having AppEngine be in an unreliable state isn't helping my cause at all. Derrick On Wed, Oct 27, 2010 at 12:25 PM, Carlos Ble wrote: > Same here. We are just planning a migration out of GAE :-(

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-10-27 Thread Carlos Ble
Same here. We are just planning a migration out of GAE :-( On 27 oct, 19:44, johnP wrote: > Appengine has been horrible since the last maintenance.  Periodic > waves of "Request was aborted after waiting too long..." and > DeadlineExceededErrors...  Flurries of 25-100 of these in a 2-3 minute > p

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-10-27 Thread johnP
Appengine has been horrible since the last maintenance. Periodic waves of "Request was aborted after waiting too long..." and DeadlineExceededErrors... Flurries of 25-100 of these in a 2-3 minute period. Occurs on simple pages as well as db-intensive ones. I'm using Django Appengine Patch. It

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-10-27 Thread derrickschneider
I wanted to chime in. We're seeing high latency (10s, 0 cpu_ms) on tasks that are put into the queue on an hourly basis. This has been going on for a couple of weeks at least. The end result is that we're churning through our CPU time early in the morning, and then our app is dead for the real work

Re: [google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-10-19 Thread Ikai Lan (Google)
Is the main page doing something that would take a long time to load? Multiple datastore calls, URLFetches? -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-10-19 Thread Stepan
My app id pixycity, it's unusable for last 4 days, cold starts end regularly in DeadlineExceededError. One week ago the cold starts took long but were successful. Thanks Stepan www.pixycity.com -- You received this message because you are subscribed to the Google Groups "Google App Engine" group

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-10-05 Thread John Gardner
I just saw a series of deadline exceeded errors. Our app is "cxiongame". I was essentially the only user on the system, but I was creating a number of TaskQueue jobs. I assume the failures were due to spin up time on additional instances to handle the task queues. It would be nice if there was

Re: [google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-29 Thread Ikai Lan (Google)
I believe the mail issues may be separate issues. I'll look into those. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Wed, Sep 29, 2010 at 5:05 PM, Jason

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-29 Thread Jason C
We (appid: steprep) still see bursts of DeadlineExceededErrors (e.g., log time: 7.29a and 8.24a, Sep 29), but this is a different characteristic than before where we had _ongoing_ 10-s and 30-s timeouts on cron jobs. We also have been seeing the mail.Send() timeout issues for the past couple of da

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-29 Thread Carlos Ble
We are still facing as many DeadlineExceededError as the past days, but now it is happening while sending emails: DeadlineExceededError: The API call mail.Send() took too long to respond and was cancelled So the app loads, it works and it gets this exception just trying to send a single email to

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-28 Thread Jason C
As of 7.28p (log time) on Tuesday night, my logs show a massive improvement. MASSIVE. I almost cannot believe the difference. Did something change? Man, oh man I hope this isn't just a time of day anomaly... This. rocks. hard. j On Sep 27, 4:55 pm, Jason C wrote: > Appid: steprep, python, QPS

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-27 Thread Jason C
Appid: steprep, python, QPS around 9-10 We're still seeing large error rates on requests (primarily cron/ taskqueue), seemingly to do with autoscaling. Here's an example chunk: 09-27 03:41PM 22.306 /_ah/queue/deferred 500 35908ms 09-27 03:41PM 48.096 /_ah/queue/deferred 500 10028ms 09-27 03:41PM

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-26 Thread Robin
I have seen this just once on Java. The app ID is visualwebflows. All custom code, using the datastore directly without an ORM. Only happened once, but the site is in development so not seeing more than 100 hits a day. I have redacted the URLs used, but below is a stack trace from the error log.

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-25 Thread Raymond C.
My app is still having a lot of timeout on DB put and get. I am just tired to keep reporting here while no one seems to be doing anything about it. I miss the days before the maintenance before 13/9. On Sep 25, 5:42 am, Jason C wrote: > This thread is somewhat quiet today, but we (appid steprep

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-25 Thread James Saull
I had a few more today: 09-25 05:58AM 51.411 /food/view 500 10020ms 0cpu_ms 0kb Mozilla/5.0 09-25 05:57AM 55.829 /food/view 500 49554ms 0cpu_ms 0kb Mozilla/5.0 Ikai - my app is in test at the moment and my config requires login/ admin on all pages. My QPS therefore is nearly 0 and the problem may

Re: [google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-25 Thread Carlos Ble
We are having just a few visitors today but still got some 500 codes. How can I know the QPS? 2010/9/24 Jason C > This thread is somewhat quiet today, but we (appid steprep) are still > seeing lots of time outs. Running at about 7 QPS when this happened: > > 09-24 02:37PM 57.365 /_ah/queue/defer

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-24 Thread Jason C
This thread is somewhat quiet today, but we (appid steprep) are still seeing lots of time outs. Running at about 7 QPS when this happened: 09-24 02:37PM 57.365 /_ah/queue/deferred 500 10021ms 09-24 02:37PM 51.017 /_ah/queue/deferred 500 10011ms 09-24 02:37PM 47.446 /_ah/queue/deferred 500 10013ms

Re: [google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-24 Thread Ikai Lan (Google)
Thanks James, that's really useful information. Did you post how many QPS your application does? -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Fri, Sep 2

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-24 Thread James Saull
My site does appear to be fine now and has been error free for at least the last 24 hours. But like I said, my app receives no traffic except for me testing it so I could just be visiting at lucky times. Let me know if you would like to know my appspot application id and if you want me to add diagn

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-24 Thread James Saull
Even on Python pages using Django templates (such my "About" page that did not access anything (e.g. memcache or db) I still had the dreaded failures. Sporadic too. I'd be clicking around the app briskly with no issues and then a few minutes later it would just return 500. All URLs on my site requi

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-24 Thread John Gardner
I just saw a rash of timeouts (actually, Download Error 5 from an urlfetch, which appears to be DEADLINE_EXCEEDED). This was after 5 seconds or so, in a task queue operation that does nothing apart from a single urlfetch. The timings logged for the request are all around 5 seconds: ms=5276 cpu_m

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-24 Thread Blixt
Our application (simpleeditionsapp) is built with Python and makes use of the Django 1.1 library. We get a lot of DeadlineExceededErrors during imports with loading requests. We have been getting these errors sporadically for the last week. On Sep 23, 11:13 pm, Fabiant7t wrote: > @Ikai > We do ha

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-23 Thread Fabiant7t
@Ikai We do have about 1.2 requests/second on our application with the ID "dctptv", so that's probably a good app to debug the problem with. The application is written in Python, using Memcache, URL Fetch, TaskQueue APIs, the Datastore and a zip-imported version of Django 1.2.3. The whole app has

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-23 Thread Darien Caldwell
all serving for my app is now 100% dead. even the simplest request is coming up 500 errors. App ID is slconductor. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To u

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-23 Thread Jason C
Lest you think it's something specific with deferred, I get bursts on other urls as well. My examples often have lots of deferred because that is by far our most common url. Here is another example: 09-23 01:25PM 27.093 /mgmt/search/start/ 500 29684ms 09-23 01:23PM 55.101 /mgmt/search/start/ 500 2

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-23 Thread Jason C
Yes, in the /_ah/queue/deferred requests, we are almost certainly doing datastore/memcache work and very likely doing urlfetch work (I'm referring to this probabilistically, because we use deferred for many things). BTW, appid steprep is Python. j On Sep 23, 2:21 pm, "Ikai Lan (Google)" wrote:

Re: [google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-23 Thread Ikai Lan (Google)
Jason, are you calling URLFetch/datastore/Memcache in that action? So far there are no reports of Java applications being impacted. We usually have a bit more of flexibility when it comes to tearing down and spinning up Python instances, so I'm wondering if this is related. -- Ikai Lan Developer

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-23 Thread Jason C
When I refer to bursts of errors, this is an example of what I mean. This is a query for any 500-series errors against appid: steprep. Note how close together the log entries are (i.e., the bursty-ness). These requests operate very quickly when App Engine is performing well. 09-23 01:07PM 42.459 /

[google-appengine] Re: Latency spikes and aborted requests in the last 24 hours

2010-09-23 Thread Jason C
We get lots of bursts of loading failures. Most recently: 9.45a to 9.47a log time (2010-09-23). Appid: steprep We also get lots of ongoing 10-second timeout. Using the logs, it seems to me that our user-facing requests are below the 1000ms threshold, so I don't know why we get so many 10-second ti