Re: [google-appengine] Re: Latency related follow up
Does anyone know if Google App Engine for Business also exhibit the same request aborted errors arising out of cold start issues? Thanks! On Wed, Oct 6, 2010 at 1:10 AM, Julian Namaro namarojul...@gmail.comwrote: Sounds like a cold start problem. I don't have experience on this but there are some advices in the forum. If your app grows to have sustained traffic it will improve for sure, but when starting or for low-traffic sites the problem remains. I see that there's a feature on the App Engine Roadmap to help address this: Ability to reserve instances to reduce application loading overhead. On Oct 5, 12:47 pm, Kangesh Gunaseelan kang...@gmail.com wrote: Thanks for your suggestion. I will take a look at the appstats one more time. What I am noticing though is api_cpu under 500ms (and many much below that) but sometimes overall cpu overshoots 700 ms. More over, for request aborted error messages, it doesn't really look like the service is even called - overall cpu in those instances is over 1 ms but api_cpu is 0 ms. Am I interpreting this right? I have seen old posts where users recommended a cron and I tried that out today with surprisingly positive results. Other than this potentially causing few instances to stay alive, I can't think of any other explanations. Any one why that is better? In any case, I wonder if that is going to really help if concurrent requests increase and demand more active instances. Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Latency related follow up
Sounds like a cold start problem. I don't have experience on this but there are some advices in the forum. If your app grows to have sustained traffic it will improve for sure, but when starting or for low-traffic sites the problem remains. I see that there's a feature on the App Engine Roadmap to help address this: Ability to reserve instances to reduce application loading overhead. On Oct 5, 12:47 pm, Kangesh Gunaseelan kang...@gmail.com wrote: Thanks for your suggestion. I will take a look at the appstats one more time. What I am noticing though is api_cpu under 500ms (and many much below that) but sometimes overall cpu overshoots 700 ms. More over, for request aborted error messages, it doesn't really look like the service is even called - overall cpu in those instances is over 1 ms but api_cpu is 0 ms. Am I interpreting this right? I have seen old posts where users recommended a cron and I tried that out today with surprisingly positive results. Other than this potentially causing few instances to stay alive, I can't think of any other explanations. Any one why that is better? In any case, I wonder if that is going to really help if concurrent requests increase and demand more active instances. Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Latency related follow up
kg, I have been using GAE for a couple of years now and this is a highly unusual situation. GAE has been a very successful product IMHO. Whether recent events are a one-time situation or whether GAE has become a victim of its own success is hard to say. Speaking only for myself and my fledgling business, I intend to stick with it. As for what one can do, we do a fair amount of exception handling and retry in the code. When testing, we go to the extent of injecting random datastore errors and making sure the integrity of the data is preserved. It gets expensive to do so, and we don't do as much as I would like, but the investment has been worth it. HTH. On Oct 3, 10:42 pm, kg kang...@gmail.com wrote: My service launched on 9/14 just the day after 9/13 when the maintenance happened. 20% of my requests fail due to the following error today...And this just happens every day. W 10-03 07:02PM 45.207 Request was aborted after waiting too long to attempt to service your request. This may happen sporadically when the App Engine serving cluster is under unexpectedly high or uneven load. If you see this message frequently, please contact the App Engine team. I have seen Google acknowledge that the issue is still ongoing and that they are working to fix it, which is certainly good to hear. My question though is: Are these errors a fact of life on GAE or just this worse since 9/13 maintenance? It pains to me such a powerful platform ridden with these issues and kicking myself here but I am hoping to hear it is after 9/13 maintenance. Is there anything an app developer could do to mitigate this issue other than retrying (which is usually end users really doing the trying or may simply abandon your service)? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Latency related follow up
Hi, Thanks for your feedback. That is good to know. I think I am handling errors reasonably well but there is only so much an app developer can do to handle the errors. Ultimately, it is the end user that will need to resolve the error. Here is an example: When an end user tries to post a message (which talks to GAE custom service), sometimes an error occurs. I am able to handle this error by either queuing for offline sending or just letting the user to know to retry. The latter case is in and of itself is bad. Same feeling that I would have when I compose a long mail in Gmail and SEND fails and asks me to retry. Here is another scenario that I am even concerned about: I want to notify users that they have a new message. I am thinking of employing hooks from within GAE to notify users that there is a new message (via a payload that indicates the message identifier). Now, when the users try to open the app and see what the messages were, they ought to get the new messages. Well, based on the GAE errors (i.e request aborted errors), that is going to be a hit and miss. When this happens, the users are going to get annoyed and feel that they are being spammed or tricked into accessing the application. Google Folks, I understand that you are working towards resolving this. Any further updates on when you will be able to resolve this. This is probably a no but I will ask any way - as a GAE developer, is there anything that we can do to avoid Request aborted error scenarios? Are you planning on reserving instances type model to get around this? Thanks. On Mon, Oct 4, 2010 at 4:25 AM, J j.si...@earlystageit.com wrote: kg, I have been using GAE for a couple of years now and this is a highly unusual situation. GAE has been a very successful product IMHO. Whether recent events are a one-time situation or whether GAE has become a victim of its own success is hard to say. Speaking only for myself and my fledgling business, I intend to stick with it. As for what one can do, we do a fair amount of exception handling and retry in the code. When testing, we go to the extent of injecting random datastore errors and making sure the integrity of the data is preserved. It gets expensive to do so, and we don't do as much as I would like, but the investment has been worth it. HTH. On Oct 3, 10:42 pm, kg kang...@gmail.com wrote: My service launched on 9/14 just the day after 9/13 when the maintenance happened. 20% of my requests fail due to the following error today...And this just happens every day. W 10-03 07:02PM 45.207 Request was aborted after waiting too long to attempt to service your request. This may happen sporadically when the App Engine serving cluster is under unexpectedly high or uneven load. If you see this message frequently, please contact the App Engine team. I have seen Google acknowledge that the issue is still ongoing and that they are working to fix it, which is certainly good to hear. My question though is: Are these errors a fact of life on GAE or just this worse since 9/13 maintenance? It pains to me such a powerful platform ridden with these issues and kicking myself here but I am hoping to hear it is after 9/13 maintenance. Is there anything an app developer could do to mitigate this issue other than retrying (which is usually end users really doing the trying or may simply abandon your service)? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Latency related follow up
Thanks for your suggestion. I will take a look at the appstats one more time. What I am noticing though is api_cpu under 500ms (and many much below that) but sometimes overall cpu overshoots 700 ms. More over, for request aborted error messages, it doesn't really look like the service is even called - overall cpu in those instances is over 1 ms but api_cpu is 0 ms. Am I interpreting this right? I have seen old posts where users recommended a cron and I tried that out today with surprisingly positive results. Other than this potentially causing few instances to stay alive, I can't think of any other explanations. Any one why that is better? In any case, I wonder if that is going to really help if concurrent requests increase and demand more active instances. Thanks. On Mon, Oct 4, 2010 at 6:30 PM, Julian Namaro namarojul...@gmail.comwrote: Hi, as a GAE developer, is there anything that we can do to avoid Request aborted error scenarios? You can run appstats, and if your app has any request taking more than ~700ms work on optimizing it. Long requests are the first to time out during periods of high latency and are generally a source of problems in App Engine. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
