Yes I can confirm this. And again: Our application gets outages and we have to pay even more Instance Hours because of this repeating GAE bug!
Just for the record. It seems exactly the same issue that Google had last August: http://code.google.com/p/googleappengine/issues/detail?id=8004 The conclusion after weeks(!) of discussion (from #8004): Me ---- Thanks, but can you please be a bit more specific? What does "The reliability team performed a maintenance operation, and it seems that most application are back to a normal levels of loading requests." mean? - Could this happen from time to time again? - The issue last for weeks! It costs us and other people a lot of money. Why do we need to escalate such a critical issue with a 50+ mailing-thread and why isn't GAE able to monitor such bad behavior by itself? Google: ----------- App Engine is a managed service, the reliability team is constantly monitoring the overall platform health and performing maintenance operation. In this case this was apparently only impacting a small number of Java applications and not the whole platform: http://code.google.com/p/googleappengine/issues/detail?id=8004 - 20 developers starred this bug - 5 developers provided an appid Filing a production issue or contacting premier customer support are the good ways to escalate such application specific issues. Me: ----- > In this case this was apparently only impacting a small number of Java > applications and not the whole platform: > - 20 developers starred this bug Please check the average starring of Java issues lately. 20 stars seems to be significant and let extrapolate that a lot more application may be affected. Nevertheless I'm pleased this issue is resolved (at least for the moment). I hope the reliability team would be able to monitor and fix such issue much faster in the future (instead of let GAE applications feel broken for weeks) On Tue, Feb 19, 2013 at 12:04 PM, Dennis <dennisf...@gmail.com> wrote: > In my app, I'm seeing a new instance start just a few minutes after a > previous new instance started. > Also, I'm getting timeout (request process taking too long) for urls that > worked fine in the past. > The GAE system status says things are fine, but this thread tells me that > it's not just my app that is having degraded service. > > > > > On Tuesday, February 19, 2013 1:02:09 PM UTC+8, aswath wrote: >> >> >> More details >> From the logs - I am working on my appid around 18:00 hrs >> >> At 18:03:10, a instance is spawn to serve >> >> /PurchaseInvoiceSetup/PurchaseInvoiceSetup.nocache. >> >> At 18:03:14, a instance is spawn to serve >> >> /newclickmenu/clickmainmenu.js >> >> >> an instance is spawned in less than 5 seconds duration for serving another >> request. >> >> I came back to my application around 18:28. I just type my >> appid.appspot.com at the url. >> My console settings for idle instances is AUTO. I can imagine a new >> instance getting started for the above request >> >> so, at 18:29:52, a new instance is spawned for the above request >> >> / >> >> The above request is a home page, and several image requests and css >> requests are going from the browser >> But, at 18:30:00, a request >> >> /images/foot.jpg >> >> is going from the browser. It is causing a new instance to be spawned >> with in 9 secs. why? >> >> -Aswath >> >> On Mon, Feb 18, 2013 at 8:01 PM, Barry Hunter <barryb...@gmail.com> wrote: >>> >>> It might help to provide some context here. Remember nobody else 'knows' >>> the structure of your application, or what its doing. >>> >>> What exact issue are you reporting? >>> >>> To the uninitiated to the issue you think you are seeing, its not clear >>> what to look for. Looking at those logs looks pretty reasonable. >>> >>> >>> >>> (although I would think just moving to static handlers for your >>> image/js/css assets would be an immidiate benefit) >>> >>> >>> >>> On Mon, Feb 18, 2013 at 2:09 PM, Aswath Satrasala <aswath.s...@gmail.com> >>> wrote: >>>> >>>> Hi, >>>> This topic is already discussed in few threads many times >>>> >>>> I have a paid application appid vs3-accounting. This is my test server. >>>> So, I can say, I am the only user accessing the server now. >>>> Look at the warnings logs, and that says that the context is >>>> initialized. >>>> Approx, in 30 min, 4 instances are started. >>>> >>>> My Application settings in the admin console are the default settings >>>> >>>> Request the Appengine team to please provide a solution for this >>>> instance startup issue. >>>> >>>> -Aswath >>>> >>>> >>>> 2013-02-18 18:30:22.734 /images/foot.jpg 200 29373ms 0kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> W2013-02-18 18:30:13.379 com.veersoft.listener.VsServletContextListener >>>> contextInitialized: 2013-02-18 19:00:13 IST >>>> >>>> I2013-02-18 18:30:17.740 >>>> com.google.inject.internal.util.$FinalizableReferenceQueue$SystemLoader >>>> loadFinalizer: Not allowed to access system class loader. >>>> >>>> I2013-02-18 18:30:17.747 >>>> com.google.inject.internal.util.$FinalizableReferenceQueue <init>: Failed >>>> to >>>> start reference finalizer thread. Reference cleanup will only occur when >>>> >>>> I2013-02-18 18:30:22.734 This request caused a new process to be started >>>> for your application, and thus caused your application code to be loaded >>>> for >>>> the first time. This requ >>>> >>>> 2013-02-18 18:30:00.099 /images/fb.jpg 200 102ms 1kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:30:00.002 /images/powered.jpg 200 127ms 3kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:59.892 /images/sep1.jpg 200 251ms 0kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:59.820 /indicator.gif 404 115ms 0kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:59.639 /images/blog.png 200 22ms 2kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:59.626 /images/export.png 200 23ms 2kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:59.523 /images/invoice.png 200 187ms 3kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:59.258 /images/taxes.png 200 23ms 4kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:59.246 /images/ledgers.png 200 22ms 4kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:59.236 /images/reports.png 200 24ms 2kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:59.206 /images/branches.png 200 22ms 5kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:58.989 /images/invite.png 200 29ms 1kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:58.889 /images/easy.jpg 200 63ms 0kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:58.873 /images/company.png 200 23ms 2kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:58.861 /images/arrow1.gif 200 24ms 0kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:58.836 /images/try.png 200 22ms 1kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:58.605 /images/new.png 200 23ms 3kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:58.519 /images/imp.jpg 200 4404ms 3kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:58.483 /images/exp.jpg 200 4398ms 3kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:58.482 /images/login-bg.jpg 200 5112ms 0kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:58.481 /images/sep.jpg 200 5097ms 0kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:58.249 /images/accountingguru.jpg 200 4985ms 4kb >>>> Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) >>>> Chrome/24.0.1312.57 Safari/537.17 >>>> >>>> 2013-02-18 18:29:53.797 /images/blackarrow.gif 304 23ms 0kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:53.768 /images/banner.png 304 23ms 0kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:53.453 /images/call.png 304 59ms 0kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:53.391 /images/background.jpg 200 184ms 0kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:52.715 /js/gfeedfetcher.js 200 93ms 6kb Mozilla/5.0 >>>> (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:52.492 /css/style.css 304 23ms 0kb Mozilla/5.0 (Windows >>>> NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 >>>> Safari/537.17 >>>> >>>> 2013-02-18 18:29:52.056 / 200 20879ms 12kb Mozilla/5.0 (Windows NT 6.1) >>>> AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 Safari/537.17 >>>> >>>> W2013-02-18 18:29:34.946 com.veersoft.listener.VsServletContextListener >>>> contextInitialized: 2013-02-18 18:59:34 IST >>>> >>>> I2013-02-18 18:29:46.854 >>>> com.google.inject.internal.util.$FinalizableReferenceQueue$SystemLoader >>>> loadFinalizer: Not allowed to access system class loader. >>>> >>>> I2013-02-18 18:29:46.931 >>>> com.google.inject.internal.util.$FinalizableReferenceQueue <init>: Failed >>>> to >>>> start reference finalizer thread. Reference cleanup will only occur when >>>> >>>> I2013-02-18 18:29:52.056 This request caused a new process to be started >>>> for your application, and thus caused your application code to be loaded >>>> for >>>> the first time. This requ >>>> >>>> 2013-02-18 18:03:14.709 /newclickmenu/clickmainmenu.js 200 38856ms 93kb >>>> Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) >>>> Chrome/24.0.1312.57 Safari/537.17 >>>> >>>> W2013-02-18 18:02:51.018 com.veersoft.listener.VsServletContextListener >>>> contextInitialized: 2013-02-18 18:32:51 IST >>>> >>>> I2013-02-18 18:03:01.921 >>>> com.google.inject.internal.util.$FinalizableReferenceQueue$SystemLoader >>>> loadFinalizer: Not allowed to access system class loader. >>>> >>>> I2013-02-18 18:03:01.937 >>>> com.google.inject.internal.util.$FinalizableReferenceQueue <init>: Failed >>>> to >>>> start reference finalizer thread. Reference cleanup will only occur when >>>> >>>> I2013-02-18 18:03:14.708 This request caused a new process to be started >>>> for your application, and thus caused your application code to be loaded >>>> for >>>> the first time. This requ >>>> >>>> 2013-02-18 18:03:10.654 >>>> /PurchaseInvoiceSetup/PurchaseInvoiceSetup.nocache.js 200 34853ms 6kb >>>> Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) >>>> Chrome/24.0.1312.57 Safari/537.17 >>>> >>>> W2013-02-18 18:02:52.131 com.veersoft.listener.VsServletContextListener >>>> contextInitialized: 2013-02-18 18:32:52 IST >>>> >>>> I2013-02-18 18:03:00.237 >>>> com.google.inject.internal.util.$FinalizableReferenceQueue$SystemLoader >>>> loadFinalizer: Not allowed to access system class loader. >>>> >>>> I2013-02-18 18:03:00.245 >>>> com.google.inject.internal.util.$FinalizableReferenceQueue <init>: Failed >>>> to >>>> start reference finalizer thread. Reference cleanup will only occur when >>>> >>>> I2013-02-18 18:03:10.654 This request caused a new process to be started >>>> for your application, and thus caused your application code to be loaded >>>> for >>>> the first time. This requ >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Google App Engine" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to google-appengi...@googlegroups.com. >>>> To post to this group, send email to google-a...@googlegroups.com. >>>> >>>> Visit this group at >>>> http://groups.google.com/group/google-appengine?hl=en. >>>> For more options, visit https://groups.google.com/groups/opt_out. >>>> >>>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to google-appengi...@googlegroups.com. >>> To post to this group, send email to google-a...@googlegroups.com. >>> >>> Visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >>> >> >> > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
