[google-appengine] GAE/J instances not shutting down as expected

[Damith C Rajapakse]

Since a couple of days, I noticed that extra instances started to serve a 
spike in requests do not shut down as expected after the spike is over. 
This is causing us to incur extra instance hour charges. Is anyone else 
experiencing the same? 
For examples, I have two instances running for nearly an hour now, but 
there is hardly any traffic. See screenshots below.
App ID: teammatesv4
-damith 







-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] GAE/J instances not shutting down as expected

[Vinny P]

On Sun, Aug 25, 2013 at 10:12 PM, Damith C Rajapakse wrote:

> Since a couple of days, I noticed that extra instances started to serve a
> spike in requests do not shut down as expected after the spike is over.
> This is causing us to incur extra instance hour charges. Is anyone else
> experiencing the same?
> For examples, I have two instances running for nearly an hour now, but
> there is hardly any traffic.
>



Can you post a picture of the *Instances* graph as well? In the main
screen, use the dropdown box just above the default graph and select *
Instances*. Here's an example picture: http://imgur.com/NdLWKSF

In addition: how much are you currently paying, and how much were you
previously paying? What is the increase in monetary charges you feel
is from "*extra instance hour charges*"? Please also include a screenshot
of the "Billing Status" section (it's on the main screen, just below the
graphs and *Instances* table).

-
-Vinny P
Technology & Media Advisor
Chicago, IL

App Engine Code Samples: http://www.learntogoogleit.com

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] GAE/J instances not shutting down as expected

[Rafael]

same here: http://cl.ly/image/2D1B1O2C060c

5 minutes later with instances that should be dead:
http://cl.ly/image/1O0Y3m473m0f after 7 minutes:
http://cl.ly/image/042H0p442K1E then many minutes later:
http://cl.ly/image/3b1R430i2W1N


On Sun, Aug 25, 2013 at 10:51 PM, Vinny P  wrote:

> On Sun, Aug 25, 2013 at 10:12 PM, Damith C Rajapakse wrote:
>
>> Since a couple of days, I noticed that extra instances started to serve a
>> spike in requests do not shut down as expected after the spike is over.
>> This is causing us to incur extra instance hour charges. Is anyone else
>> experiencing the same?
>> For examples, I have two instances running for nearly an hour now, but
>> there is hardly any traffic.
>>
>
>
>
> Can you post a picture of the *Instances* graph as well? In the main
> screen, use the dropdown box just above the default graph and select *
> Instances*. Here's an example picture: http://imgur.com/NdLWKSF
>
> In addition: how much are you currently paying, and how much were you
> previously paying? What is the increase in monetary charges you feel
> is from "*extra instance hour charges*"? Please also include a screenshot
> of the "Billing Status" section (it's on the main screen, just below the
> graphs and *Instances* table).
>
> -
> -Vinny P
> Technology & Media Advisor
> Chicago, IL
>
> App Engine Code Samples: http://www.learntogoogleit.com
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at http://groups.google.com/group/google-appengine.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] GAE/J instances not shutting down as expected

[Damith C Rajapakse]

Thanks Vinny for the quick response. Sorry for the delay in replying. I was 
monitoring the app for a recurrence before replying.
One seems to be happening now. Given below is the screen shot.
As for billing, ours is a low traffic app which can survive on the free 
quota of 28 instance hours per day. The extra 4 hours are more than enough 
to cover spikes in usage, as long as the extra instances shutdown promptly. 
Since a few days ago, we started getting billed for instance hours 
(typically, just a few hours over the quota) without any change in traffic.
That's how I came to realize that instances don't shutdown as promptly as 
before, on some occasions. In the screenshot below, the list of instances 
doesn't even match with the graph.
The list shows that the extra instances has been up for 2 hours, but the 
graph shows only one instance.
I suspect some fine tuning of the algorithm is going on, causing some 
glitches in the process. 


 

On Monday, August 26, 2013 1:51:09 PM UTC+8, Vinny P wrote:
>
> On Sun, Aug 25, 2013 at 10:12 PM, Damith C Rajapakse wrote:
>
>> Since a couple of days, I noticed that extra instances started to serve a 
>> spike in requests do not shut down as expected after the spike is over. 
>> This is causing us to incur extra instance hour charges. Is anyone else 
>> experiencing the same? 
>> For examples, I have two instances running for nearly an hour now, but 
>> there is hardly any traffic.
>>
>  
>  
>  
> Can you post a picture of the *Instances* graph as well? In the main 
> screen, use the dropdown box just above the default graph and select *
> Instances*. Here's an example picture: http://imgur.com/NdLWKSF
>  
> In addition: how much are you currently paying, and how much were you 
> previously paying? What is the increase in monetary charges you feel 
> is from "*extra instance hour charges*"? Please also include a screenshot 
> of the "Billing Status" section (it's on the main screen, just below the 
> graphs and *Instances* table).
>   
> -
> -Vinny P
> Technology & Media Advisor
> Chicago, IL
>
> App Engine Code Samples: http://www.learntogoogleit.com
>   
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] GAE/J instances not shutting down as expected

[Vinny P]

On Tue, Aug 27, 2013 at 10:38 PM, Damith C Rajapakse 
 wrote:

> Since a few days ago, we started getting billed for instance hours
> (typically, just a few hours over the quota) without any change in traffic.
>


If the increased instance hour costs started just a few days ago, I
wouldn't worry too much. App Engine is currently in the middle of upgrading
applications from the 1.8.3 runtime to the 1.8.4 runtime, and there's
always "weirdness" that occurs during upgrades. Keep an eye on it for the
next week, it might fix itself. If it doesn't, then there are some things
that can be looked at and optimized.



-
-Vinny P
Technology & Media Advisor
Chicago, IL

App Engine Code Samples: http://www.learntogoogleit.com

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] GAE/J instances not shutting down as expected

[Cesium]

I'm also seeing new scheduler behavior since MDT 8/28/2013.

Not one instance has been torn down since then.
No user facing requests have encountered a delay due to cranking up an instance.

Sweet huh?

Can't wait to see what happens now that my memory leaks accumulate without 
being "reset."

David

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] GAE/J instances not shutting down as expected

[Cesium]

Check this out. New scheduler behavior is sweet, starting at about 1.25 
days ago.
>
>


And here's the memory usage.



An F1 instance maxed out. New instance started a few hours ago.

Lovely stuff if it lasts.

David


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] GAE/J instances not shutting down as expected

[Cesium]

Of course it did not last. 3 days of this new behavior. 1 day of reversion to 
previous characteristics, now back to the new behavior.
H.
David

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] GAE/J instances not shutting down as expected

[Aswath Satrasala]

In the admin console, I adjusted the max idle instance = 1.  This seems to
help my application.  When there is no/less traffic, the instances are
shutdown properly.

Before, my appid idle instances setting was '1- automatic'.  This setting
was not shutting down the instances when there is no traffic.  Now, with
the above setting, my cost is much more under control.


-Aswath


On Mon, Aug 26, 2013 at 8:42 AM, Damith C Rajapakse wrote:

> Since a couple of days, I noticed that extra instances started to serve a
> spike in requests do not shut down as expected after the spike is over.
> This is causing us to incur extra instance hour charges. Is anyone else
> experiencing the same?
> For examples, I have two instances running for nearly an hour now, but
> there is hardly any traffic. See screenshots below.
> App ID: teammatesv4
> -damith
>
>
> 
>
>
> 
>
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at http://groups.google.com/group/google-appengine.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.