[google-appengine] Re: Policy for instance startup
I hoped that Warmup Requests could be used to remedy the situation (at least from a users perspective). Unfortunally, it does not help anything. Warmup requests seems to be inactive (or not react) when there are less than 10 instances active for my live app. The warm up requests starts to work when 10 instances are started. It seems to me that my app-id have been marked permanently as a heavy application, which isn't true yet. On 26 Nov, 09:32, Tomas Alaeus tala...@gmail.com wrote: This problem doesn't seem to happen on the staging server (doesn't have billing enabled), but I maybe haven't stressed it enough. Another application I have for testing purposes seem to be affected by it too, and it doesn't have billing enabled either. That application seems to want at least 4 instances before serving hot instances to incoming requests. This is actually quite bad for everyone. If the load is low the user experience will be worse, we have to pay more and Googles servers will have to boot up/down instances for no reason. Can any Googler confirm this behavior? On 24 Nov, 22:18, nickmilon nickmi...@gmail.com wrote: While waiting from a Googler to give us some light on this may be you do your stress tests in the staging server ? (just an other version will do as staging server) Also I remember people from App Engine team stating here that the best performance comes when load comes in a natural matter. Still IMHO that there must be some more transparency into those parameters within which our appls have to operate. - It helps both us and the platform. Regards Nick On Nov 24, 9:21 am, Tomas Alaeus tala...@gmail.com wrote: Probably. With some more thinking I got to this conclusion: At first, an application can utilize up to 30 instances. After having used all those instances for a few minutes, and it still isn't enough, GAE will increase the maximum number of available instances to 40, and then to 50 etc. Every time it increases the number of instances it will probably also adjust how and when it boots up new instances. For my production app it probably tries to keep at least 10 instances running since it thinks a huge amount of users can use the site at any given time. The staging haven't had any rush of users, and therefore thinks it is safe with just 1 instance running. I would still like to hear something from a Googler. If this is how it works then it means that stress testing the application (with a lot more pressure than it will normally get) may lead to worse (!) performance permanently. On 24 Nov, 00:01, nickmilon nickmi...@gmail.com wrote: Not been a Googler can't help much with this. Having said that, I suspect there is a kind of build in algorithm that does some kind of application profiling taking into acount QPS, response times, and other parameters which adjusts instance life time, number of instances to start etc.. This could possibly explain the difference in behaviour between your staging and production appls. happy coding;-) On Nov 23, 11:58 am, Tomas Alaeus tala...@gmail.com wrote: I'm curious when exactly instances are started. I have two applications running on GAE, one of them have billing enabled. The one with billing enabled have been stress tested and have at most started 100 simultaneous instances. The other is just for testing and staging purposes and have never handeled much traffic. What I experience is that the staging server never starts more instances than needed. If a single person views pages it will never load more than a single instance. The other one however seems to start about 5 instances before anyone can get hot responses, and it will continue to start up to about 10 before realizing that ~1 QPS isn't that much traffic (the requests finish in about 100ms each). So, why does GAE boot up lots of instances even though 1 instance can serve the incoming traffic without a problem (the requests doesn't even overlap, so no waiting is needed)? I realize that this isn't a very big issue, since when it gets lots of traffic it will indeed need all the instances. I'm just curious why it happens. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Policy for instance startup
This problem doesn't seem to happen on the staging server (doesn't have billing enabled), but I maybe haven't stressed it enough. Another application I have for testing purposes seem to be affected by it too, and it doesn't have billing enabled either. That application seems to want at least 4 instances before serving hot instances to incoming requests. This is actually quite bad for everyone. If the load is low the user experience will be worse, we have to pay more and Googles servers will have to boot up/down instances for no reason. Can any Googler confirm this behavior? On 24 Nov, 22:18, nickmilon nickmi...@gmail.com wrote: While waiting from a Googler to give us some light on this may be you do your stress tests in the staging server ? (just an other version will do as staging server) Also I remember people from App Engine team stating here that the best performance comes when load comes in a natural matter. Still IMHO that there must be some more transparency into those parameters within which our appls have to operate. - It helps both us and the platform. Regards Nick On Nov 24, 9:21 am, Tomas Alaeus tala...@gmail.com wrote: Probably. With some more thinking I got to this conclusion: At first, an application can utilize up to 30 instances. After having used all those instances for a few minutes, and it still isn't enough, GAE will increase the maximum number of available instances to 40, and then to 50 etc. Every time it increases the number of instances it will probably also adjust how and when it boots up new instances. For my production app it probably tries to keep at least 10 instances running since it thinks a huge amount of users can use the site at any given time. The staging haven't had any rush of users, and therefore thinks it is safe with just 1 instance running. I would still like to hear something from a Googler. If this is how it works then it means that stress testing the application (with a lot more pressure than it will normally get) may lead to worse (!) performance permanently. On 24 Nov, 00:01, nickmilon nickmi...@gmail.com wrote: Not been a Googler can't help much with this. Having said that, I suspect there is a kind of build in algorithm that does some kind of application profiling taking into acount QPS, response times, and other parameters which adjusts instance life time, number of instances to start etc.. This could possibly explain the difference in behaviour between your staging and production appls. happy coding;-) On Nov 23, 11:58 am, Tomas Alaeus tala...@gmail.com wrote: I'm curious when exactly instances are started. I have two applications running on GAE, one of them have billing enabled. The one with billing enabled have been stress tested and have at most started 100 simultaneous instances. The other is just for testing and staging purposes and have never handeled much traffic. What I experience is that the staging server never starts more instances than needed. If a single person views pages it will never load more than a single instance. The other one however seems to start about 5 instances before anyone can get hot responses, and it will continue to start up to about 10 before realizing that ~1 QPS isn't that much traffic (the requests finish in about 100ms each). So, why does GAE boot up lots of instances even though 1 instance can serve the incoming traffic without a problem (the requests doesn't even overlap, so no waiting is needed)? I realize that this isn't a very big issue, since when it gets lots of traffic it will indeed need all the instances. I'm just curious why it happens. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Policy for instance startup
While waiting from a Googler to give us some light on this may be you do your stress tests in the staging server ? (just an other version will do as staging server) Also I remember people from App Engine team stating here that the best performance comes when load comes in a natural matter. Still IMHO that there must be some more transparency into those parameters within which our appls have to operate. - It helps both us and the platform. Regards Nick On Nov 24, 9:21 am, Tomas Alaeus tala...@gmail.com wrote: Probably. With some more thinking I got to this conclusion: At first, an application can utilize up to 30 instances. After having used all those instances for a few minutes, and it still isn't enough, GAE will increase the maximum number of available instances to 40, and then to 50 etc. Every time it increases the number of instances it will probably also adjust how and when it boots up new instances. For my production app it probably tries to keep at least 10 instances running since it thinks a huge amount of users can use the site at any given time. The staging haven't had any rush of users, and therefore thinks it is safe with just 1 instance running. I would still like to hear something from a Googler. If this is how it works then it means that stress testing the application (with a lot more pressure than it will normally get) may lead to worse (!) performance permanently. On 24 Nov, 00:01, nickmilon nickmi...@gmail.com wrote: Not been a Googler can't help much with this. Having said that, I suspect there is a kind of build in algorithm that does some kind of application profiling taking into acount QPS, response times, and other parameters which adjusts instance life time, number of instances to start etc.. This could possibly explain the difference in behaviour between your staging and production appls. happy coding;-) On Nov 23, 11:58 am, Tomas Alaeus tala...@gmail.com wrote: I'm curious when exactly instances are started. I have two applications running on GAE, one of them have billing enabled. The one with billing enabled have been stress tested and have at most started 100 simultaneous instances. The other is just for testing and staging purposes and have never handeled much traffic. What I experience is that the staging server never starts more instances than needed. If a single person views pages it will never load more than a single instance. The other one however seems to start about 5 instances before anyone can get hot responses, and it will continue to start up to about 10 before realizing that ~1 QPS isn't that much traffic (the requests finish in about 100ms each). So, why does GAE boot up lots of instances even though 1 instance can serve the incoming traffic without a problem (the requests doesn't even overlap, so no waiting is needed)? I realize that this isn't a very big issue, since when it gets lots of traffic it will indeed need all the instances. I'm just curious why it happens. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Policy for instance startup
Not been a Googler can't help much with this. Having said that, I suspect there is a kind of build in algorithm that does some kind of application profiling taking into acount QPS, response times, and other parameters which adjusts instance life time, number of instances to start etc.. This could possibly explain the difference in behaviour between your staging and production appls. happy coding;-) On Nov 23, 11:58 am, Tomas Alaeus tala...@gmail.com wrote: I'm curious when exactly instances are started. I have two applications running on GAE, one of them have billing enabled. The one with billing enabled have been stress tested and have at most started 100 simultaneous instances. The other is just for testing and staging purposes and have never handeled much traffic. What I experience is that the staging server never starts more instances than needed. If a single person views pages it will never load more than a single instance. The other one however seems to start about 5 instances before anyone can get hot responses, and it will continue to start up to about 10 before realizing that ~1 QPS isn't that much traffic (the requests finish in about 100ms each). So, why does GAE boot up lots of instances even though 1 instance can serve the incoming traffic without a problem (the requests doesn't even overlap, so no waiting is needed)? I realize that this isn't a very big issue, since when it gets lots of traffic it will indeed need all the instances. I'm just curious why it happens. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Policy for instance startup
Probably. With some more thinking I got to this conclusion: At first, an application can utilize up to 30 instances. After having used all those instances for a few minutes, and it still isn't enough, GAE will increase the maximum number of available instances to 40, and then to 50 etc. Every time it increases the number of instances it will probably also adjust how and when it boots up new instances. For my production app it probably tries to keep at least 10 instances running since it thinks a huge amount of users can use the site at any given time. The staging haven't had any rush of users, and therefore thinks it is safe with just 1 instance running. I would still like to hear something from a Googler. If this is how it works then it means that stress testing the application (with a lot more pressure than it will normally get) may lead to worse (!) performance permanently. On 24 Nov, 00:01, nickmilon nickmi...@gmail.com wrote: Not been a Googler can't help much with this. Having said that, I suspect there is a kind of build in algorithm that does some kind of application profiling taking into acount QPS, response times, and other parameters which adjusts instance life time, number of instances to start etc.. This could possibly explain the difference in behaviour between your staging and production appls. happy coding;-) On Nov 23, 11:58 am, Tomas Alaeus tala...@gmail.com wrote: I'm curious when exactly instances are started. I have two applications running on GAE, one of them have billing enabled. The one with billing enabled have been stress tested and have at most started 100 simultaneous instances. The other is just for testing and staging purposes and have never handeled much traffic. What I experience is that the staging server never starts more instances than needed. If a single person views pages it will never load more than a single instance. The other one however seems to start about 5 instances before anyone can get hot responses, and it will continue to start up to about 10 before realizing that ~1 QPS isn't that much traffic (the requests finish in about 100ms each). So, why does GAE boot up lots of instances even though 1 instance can serve the incoming traffic without a problem (the requests doesn't even overlap, so no waiting is needed)? I realize that this isn't a very big issue, since when it gets lots of traffic it will indeed need all the instances. I'm just curious why it happens. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
