Jason, what are you doing in your handler that is hitting DeadlineExceededError? It looks like you're doing an URLFetch. Could the resources you are fetching be slow?
On Thu, Sep 2, 2010 at 8:38 AM, Jason C <jason.a.coll...@gmail.com> wrote: > I should have mentioned that in addition to the 30s DEEs, we are also > seeing a number of the 10s "Request was aborted" messages (the ones > that occur before any user code executes). > > j > > On Sep 2, 9:32 am, Jason C <jason.a.coll...@gmail.com> wrote: > > It may have settled down for you Tim, but we are being hammered by > > DeadlineExceededError right now. Log time: 8.27a Thursday 2010-09-02 > > > > AppID: steprep > > > > In the past, this problem seems to be localized to small (presumably) > > clusters of apps. > > > > Can someone please look into our specific cluster? > > > > Thanks, > > j > > > > On Sep 1, 6:06 pm, Tim Hoffman <zutes...@gmail.com> wrote: > > > > > > > > > Hi > > > > > It seems that the massive increase startup times has settled down. > > > > > But we effectively had to app instances down for about 16 hours due to > > > the issue. > > > It is clear the startup peformance issue is not my code as it was > > > affecting a other people, but > > > was not widespread enough to get on the system status radar. > > > > > And issue for it was raisedhttp:// > code.google.com/p/googleappengine/issues/detail?id=3667 > > > > > The outage was not good but to be honest the biggest problem my > > > management hand with it > > > was the complete absence of any response from google during this > > > period. > > > > > Last time we had a major issue it took a week, and a large report > > > prepared by myself detailing what was > > > going on before I could get any response from google. The response was > > > also > > > limited to "This is a known issue, we are working to address it" which > > > is fine in it self at least some communication > > > was eventually made but it should not have taken a week and about 3 > > > days of effort on my part to actually get any form of response. > > > > > See the final part of that saga here. > http://groups.google.com.au/group/google-appengine/browse_thread/thre... > > > > > I am huge advocate of appengine and we are using it more and more, but > > > the total feeling of helplessness you > > > get when you have instances non functioning because they can't even > > > startup for an extended period and > > > no feedback what so ever makes it very hard to even manage > > > expectations, and impossible to report to management > > > about any form of progress in resolving the issue. > > > > > In addition when this happened last time, a journalist from the > > > Register contacted me about the issue and asked for more information > > > so they could do a story about the lack of communication from google > > > on these sorts of issues. I declined, but maybe it does need some > > > more visibility and least with in the app engine user community. > > > > > Regards > > > > > Tim Hoffman > > > > > On Sep 1, 11:54 pm, Tim Hoffman <zutes...@gmail.com> wrote: > > > > > > The rate of errors has dropped off somewhat. > > > > > > It is still happening though. > > > > > > T > > > > > > On Sep 1, 7:24 pm, Tim Hoffman <zutes...@gmail.com> wrote: > > > > > > > Further investigation shows these excessive startup times are being > > > > > experienced by more than one instance. > > > > > > > Appid's qtrack-dev, q-tracker. > > > > > > > T > > > > > > > On Sep 1, 5:15 pm, Tim Hoffman <zutes...@gmail.com> wrote: > > > > > > > > Hi > > > > > > > > I am still having trouble getting instances to get past initial > > > > > > import (appid q-tracker) > > > > > > > > I have even tried starting old versions to get around failed > imports. > > > > > > > > for the last 10 minutes all startup attempts end in > DeadlineExceeded > > > > > > error. > > > > > > > > T > > > > > > > > On Sep 1, 2:45 pm, Tim Hoffman <zutes...@gmail.com> wrote: > > > > > > > > > Hi > > > > > > > > > As I have pointed out in the past its quite possible to get > pockets > > > > > > > of appengine instances experiencing extremely poor performance > and > > > > > > > this doesn't show up in the system status. (http:// > > > > > > > > groups.google.com.au/group/google-appengine/browse_thread/thread/ > > > > > > > 23e988d494144242/8ff964f7ffcc0b37?q=#8ff964f7ffcc0b37) > > > > > > > > > Apps seemd to be sharded into different parts of appengine > > > > > > > infrastructure based on appid. > > > > > > > > > I hope this problem is just transitory for tosay and I don't > have to > > > > > > > spend days collecting evidence again. > > > > > > > > > T > > > > > > > > > On Sep 1, 2:30 pm, Tim Hoffman <zutes...@gmail.com> wrote: > > > > > > > > > > Hi > > > > > > > > > > Except it isn't. I am still getting hit with extremeley long > startup > > > > > > > > times. > > > > > > > > I am not even getting around to accessing the datastore. > Failure is > > > > > > > > always during instance startup. > > > > > > > > > > T > > > > > > > > > > On Sep 1, 2:27 pm, Andi Albrecht < > albrecht.a...@googlemail.com> wrote: > > > > > > > > > > > Sorry, I've missed this announcement on Downtime Notify: > > > > > > > > > > > > http://groups.google.com/group/google-appengine-downtime-notify/brows... > > > > > > > > > > > As noted in my previous post, everything's fine now :) > > > > > > > > > > > Andi > > > > > > > > > > > Tim Hoffman <zutes...@gmail.com> writes: > > > > > > > > > > Hi > > > > > > > > > > > > I am still seeing them on and off, everything might be > fine for 20min, > > > > > > > > > > then bam. > > > > > > > > > > I have even been having some dashboard failures. > > > > > > > > > > > > T > > > > > > > > > > > > On Sep 1, 1:55 pm, Andi Albrecht < > albrecht.a...@googlemail.com> wrote: > > > > > > > > > >> Tim Hoffman <zutes...@gmail.com> writes: > > > > > > > > > >> > Hi > > > > > > > > > > > >> > As of today I am getting a lot of DeadlineExceeded > Errors on a couple > > > > > > > > > >> > of instances druing startup. > > > > > > > > > > > >> > I am not even getting past the initial imports, > > > > > > > > > > > >> > Nothing is being reported in system status, so it's > probably > > > > > > > > > >> > restricted part of the appengine structure. > > > > > > > > > > > >> > Tim > > > > > > > > > > > >> Hi Tim, > > > > > > > > > > > >> do you still see these issues? I've seen a couple of > different errors a > > > > > > > > > >> few hours ago, but now everything looks fine again. > > > > > > > > > > > >> Andi > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
