We (appid: steprep) still see bursts of DeadlineExceededErrors (e.g., log time: 7.29a and 8.24a, Sep 29), but this is a different characteristic than before where we had _ongoing_ 10-s and 30-s timeouts on cron jobs.
We also have been seeing the mail.Send() timeout issues for the past couple of days, both on appid steprep and appid daily-deal. Most of our emails are sent with taskqueue (and thus they retry); yesterday we saw some duplicate emails being delivered, so perhaps even when the mail.Send() fails, there is actually a message being delivered (which is also not good). j On Sep 29, 3:07 am, Carlos Ble <ble.jur...@gmail.com> wrote: > We are still facing as many DeadlineExceededError as the past days, > but now it is happening while sending emails: > > DeadlineExceededError: The API call mail.Send() took too long to > respond and was cancelled > > So the app loads, it works and it gets this exception just trying to > send a single email to a single recipient. Appid: mavencharts > > Thanks > > On 29 sep, 03:32, Jason C <jason.a.coll...@gmail.com> wrote: > > > > > As of 7.28p (log time) on Tuesday night, my logs show a massive > > improvement. MASSIVE. > > > I almost cannot believe the difference. > > > Did something change? Man, oh man I hope this isn't just a time of day > > anomaly... > > > This. rocks. hard. > > > j > > > On Sep 27, 4:55 pm, Jason C <jason.a.coll...@gmail.com> wrote: > > > > Appid: steprep, python, QPS around 9-10 > > > > We're still seeing large error rates on requests (primarily cron/ > > > taskqueue), seemingly to do with autoscaling. Here's an example chunk: > > > > 09-27 03:41PM 22.306 /_ah/queue/deferred 500 35908ms > > > 09-27 03:41PM 48.096 /_ah/queue/deferred 500 10028ms > > > 09-27 03:41PM 28.501 /_ah/queue/deferred 500 29527ms > > > 09-27 03:41PM 46.634 /_ah/queue/deferred 500 10034ms > > > 09-27 03:41PM 20.134 /_ah/queue/deferred 500 35703ms > > > 09-27 03:41PM 20.146 /_ah/queue/deferred 500 35691ms > > > 09-27 03:41PM 18.991 /mgmt/jobs/put/deferred/? > > > jobId=6c7dae4c-7c6a-4290-96b7-8cae4bd66410 500 36756ms > > > 09-27 03:41PM 45.607 /_ah/queue/deferred 500 10013ms > > > 09-27 03:41PM 45.297 /mgmt/jobs/put/deferred/?jobId=c5f66519- > > > f37b-4c06-87c8-ac1fe3db6633 500 10040ms > > > 09-27 03:41PM 44.434 /_ah/queue/deferred 500 10040ms > > > 09-27 03:41PM 43.555 /_ah/queue/deferred 500 10026ms > > > 09-27 03:41PM 43.236 /_ah/queue/deferred 500 10029ms > > > 09-27 03:41PM 43.230 /_ah/queue/deferred 500 10005ms > > > 09-27 03:41PM 42.436 /_ah/queue/deferred 500 10013ms > > > 09-27 03:41PM 22.305 /_ah/queue/deferred 500 29862ms > > > 09-27 03:41PM 41.949 /_ah/queue/deferred 500 10040ms > > > 09-27 03:41PM 22.174 /_ah/queue/deferred 500 29437ms > > > 09-27 03:41PM 41.446 /_ah/queue/deferred 500 10006ms > > > 09-27 03:41PM 41.308 /_ah/queue/deferred 500 10041ms > > > 09-27 03:41PM 41.188 /_ah/queue/deferred 500 10008ms > > > 09-27 03:41PM 40.760 /_ah/queue/deferred 500 10050ms > > > 09-27 03:41PM 40.551 /_ah/queue/deferred 500 10132ms > > > 09-27 03:41PM 09.059 /_ah/queue/deferred 500 41582ms > > > 09-27 03:41PM 21.013 /mgmt/search/start/ 500 29442ms > > > 09-27 03:41PM 39.855 /_ah/queue/deferred 500 10020ms > > > > Ikai, please do let me know if there's anything I can do to help from > > > my end. > > > > Thanks, > > > j > > > > On Sep 26, 2:43 am, Robin <robinmess...@gmail.com> wrote: > > > > > I have seen this just once on Java. The app ID is visualwebflows. All > > > > custom code, using the datastore directly without an ORM. > > > > > Only happened once, but the site is in development so not seeing more > > > > than 100 hits a day. > > > > > I have redacted the URLs used, but below is a stack trace from the > > > > error log. > > > > > Robin Message > > > > > [23/Sep/2010:05:58:23 -0700] "GET REDACTED HTTP/1.1" 500 0 - "Mozilla/ > > > > 5.0 (X11; U; Linux x86_64; en-GB; rv:1.9.1.9) Gecko/20100330 Fedora/ > > > > 3.5.9-1.fc11 Firefox/3.5.9,gzip(gfe)" "visualwebflows.appspot.com" > > > > ms=30574 cpu_ms=706 api_cpu_ms=64 cpm_usd=0.019645 pending_ms=232 > > > > exit_code=104 > > > > > # > > > > W 09-23 05:58AM 23.264 > > > > > Error for REDACTED > > > > com.google.apphosting.runtime.HardDeadlineExceededError: This request > > > > (322478a37b051ea5) started at 2010/09/23 12:57:52.952 UTC and was > > > > still executing at 2010/09/23 12:58:22.751 UTC. > > > > at > > > > com.google.appengine.runtime.Request.process-322478a37b051ea5(Request.java) > > > > at java.io.FileInputStream.readBytes(Native Method) > > > > at java.io.FileInputStream.read(FileInputStream.java:231) > > > > at sun.misc.Resource.getBytes(Resource.java:114) > > > > at java.net.URLClassLoader.defineClass(URLClassLoader.java:273) > > > > at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) > > > > at > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImp > > > > l.java: > > > > 43) > > > > at java.lang.reflect.Method.invoke(Method.java:616) > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:268) > > > > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:336) > > > > at > > > > net.thetadesign.cherry.server.EditServlet.getOutputFormatter(EditServlet.ja > > > > va: > > > > 215) > > > > at > > > > net.thetadesign.cherry.server.EditServlet.evaluate(EditServlet.java: > > > > 138) > > > > at > > > > net.thetadesign.cherry.server.EditServlet.render(EditServlet.java: > > > > 108) > > > > at > > > > net.thetadesign.cherry.server.EditServlet.render(EditServlet.java: > > > > 50) > > > > at > > > > net.thetadesign.cherry.server.AbstractServlet.showController(AbstractServle > > > > t.java: > > > > 176) > > > > at > > > > net.thetadesign.cherry.server.AbstractServlet.doGet(AbstractServlet.java: > > > > 144) > > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:693) > > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) > > > > at > > > > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: > > > > 511) > > > > at org.mortbay.jetty.servlet.ServletHandler > > > > $CachedChain.doFilter(ServletHandler.java:1166) > > > > at > > > > com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlo > > > > bUploadFilter.java: > > > > 97) > > > > at org.mortbay.jetty.servlet.ServletHandler > > > > $CachedChain.doFilter(ServletHandler.java:1157) > > > > at > > > > com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionF > > > > ilter.java: > > > > 35) > > > > at org.mortbay.jetty.servlet.ServletHandler > > > > $CachedChain.doFilter(ServletHandler.java:1157) > > > > at > > > > com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(Trans > > > > actionCleanupFilter.java: > > > > 43) > > > > at org.mortbay.jetty.servlet.ServletHandler > > > > $CachedChain.doFilter(ServletHandler.java:1157) > > > > at > > > > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java: > > > > 388) > > > > at > > > > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java: > > > > 216) > > > > at > > > > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java: > > > > 182) > > > > at > > > > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java: > > > > 765) > > > > at > > > > org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java: > > > > 418) > > > > at > > > > com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionH > > > > andlerMap.java: > > > > 238) > > > > at > > > > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java: > > > > 152) > > > > at org.mortbay.jetty.Server.handle(Server.java:326) > > > > at > > > > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java: > > > > 542) > > > > at org.mortbay.jetty.HttpConnection > > > > $RequestHandler.headerComplete(HttpConnection.java:923) > > > > at > > > > com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequ > > > > estParser.java: > > > > 76) > > > > at > > > > org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) > > > > at > > > > com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceReques > > > > t(JettyServletEngineAdapter.java: > > > > 135) > > > > at > > > > com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java: > > > > 251) > > > > at com.google.apphosting.base.RuntimePb$EvaluationRuntime > > > > $6.handleBlockingRequest(RuntimePb.java:6784) > > > > at com.google.apphosting.base.RuntimePb$EvaluationRuntime > > > > $6.handleBlockingRequest(RuntimePb.java:6782) > > > > at > > > > com.google.net.rpc.impl.BlockingApplicationHandler.handleRequest(BlockingAp > > > > plicationHandler.java: > > > > 24) > > > > at > > > > com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java: > > > > 398) > > > > at com.google.net.rpc.impl.Server$2.run(Server.java:852) > > > > at > > > > com.google.tracing.LocalTraceSpanRunnable.run(LocalTraceSpanRunnable.java: > > > > 56) > > > > at > > > > com.google.tracing.LocalTraceSpanBuilder.internalContinueSpan(LocalTraceSpa > > > > nBuilder.java: > > > > 576) > > > > at com.google.net.rpc.impl.Server.startRpc(Server.java:807) > > > > at > > > > com.google.net.rpc.impl.Server.processRequest(Server.java:369) > > > > at > > > > com.google.net.rpc.impl.ServerConnection.messageReceived(ServerConnection.j > > > > ava: > > > > 442) > > > > at > > > > com.google.net.rpc.impl.RpcConnection.parseMessages(RpcConnection.java: > > > > 319) > > > > at > > > > com.google.net.rpc.impl.RpcConnection.dataReceived(RpcConnection.java: > > > > 290) > > > > at > > > > com.google.net.async.Connection.handleReadEvent(Connection.java: > > > > 474) > > > > at > > > > com.google.net.async.EventDispatcher.processNetworkEvents(EventDispatcher.j > > > > ava: > > > > 831) > > > > at > > > > com.google.net.async.EventDispatcher.internalLoop(EventDispatcher.java: > > > > 207) > > > > at > > > > com.google.net.async.EventDispatcher.loop(EventDispatcher.java: > > > > 103) > > > > at > > > > com.google.net.rpc.RpcService.runUntilServerShutdown(RpcService.java: > > > > 251) > > > > at com.google.apphosting.runtime.JavaRuntime > > > > $RpcRunnable.run(JavaRuntime.java:418) > > > > at java.lang.Thread.run(Thread.java:636) -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
