Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-27 Thread Johan Euphrosine
Actually the latter knobs I listed were suggestion from the community but feel free to raise a different voice. The only knobs Google provide so far are: - maximum number of idle instances - minimum pending latency See http://code.google.com/appengine/docs/adminconsole/performancesettings.html fo

[google-appengine] Re: GAE starting unnecessary instances

2011-07-27 Thread pdknsk
It seems like the service is moving in a direction in which devs will have to adjust many knobs, rather than have Google handle it automagically in the background. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-27 Thread Johan Euphrosine
The gist of the discussion is: - Idle Dynamic Instances get traffic in priority over Idle Always On Instances. - People have shown in the thread that 'Max Idle Instance' knob could be used to workaround this behavior by preventing too many Dynamic Instances from spawning. - This is not really worth

[google-appengine] Re: GAE starting unnecessary instances

2011-07-26 Thread Santiago Lema
To keep things simple, is this issue worth worrying about before the new billing model becomes active ? Or will everything be so different that the stuff we see know will be different then ? My instances: https://skitch.com/smalltech/fcyfd/instances-smallte.ch Site: smallte.ch (on http://websmal

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-26 Thread Johan Euphrosine
On Fri, Jul 22, 2011 at 5:37 PM, Rob Coops wrote: > 1/ Idle Always On instance > Spawning a new Dynamic instance > 2/ Spawning a new Dynamic instance > Busy Always On instance > 3/ Idle Dynamic instance > Busy Always On instance > 4/ Idle Dynamic instance > Idle Always On instance > So App engine

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-26 Thread Johan Euphrosine
On Mon, Jul 25, 2011 at 9:25 PM, Galoch wrote: > > Hi Johan, > > Thanks for following up on this discussion. > > I have new findings to share with you since GAE now works a bit > differently with all these recent changes. Before I do that I will > share the changes I made to our app which has help

[google-appengine] Re: GAE starting unnecessary instances

2011-07-25 Thread Galoch
Hi Johan, Thanks for following up on this discussion. I have new findings to share with you since GAE now works a bit differently with all these recent changes. Before I do that I will share the changes I made to our app which has helped us mitigate some performance issues. 1. We moved all our

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-25 Thread Johan Euphrosine
Hi Francois, I think the help text of the 'Idle Instances' settings is pretty self-explanatory: """ You will not be charged for instances over the specified maximum. """ I can investigate on why these instances are created even thought you setup a high min pending latency, and a a Max idle instan

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-25 Thread Francois Masurel
Something strange : GAE keeps starting new dynamic instances though I have set the Min Pending Latency to 15s via the Application Settings page. My requests are served in a few ms on average and I have multithreading enabled. These instances are immediately destroyed as I have set the Max Idle

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-25 Thread Francois Masurel
Hi Johan, Thanx for your answer. Setting MaxIdleInstances to 3 did force the resident instances to be used instead of the dynamic ones. As we will soon pay for active instances, I would haved like to have the 3 resident ones being used instead of extra dynamic ones being started. Thanx again

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-25 Thread Johan Euphrosine
What is your setting for 'Max Idle Instances' ? Feel free to open a Production issue with you appid if you want me to track this specifically. On Sat, Jul 23, 2011 at 11:01 AM, Francois Masurel wrote: > Geez, now even some Dynamic instances are not used too and stay idle for > hours (check attac

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-25 Thread Johan Euphrosine
I think Ggegd covered this in his pricing FAQ: Q: How will Always On work under the new model? A: When App Engine leaves preview all Paid Apps and Apps in Premier Accounts will be able to set the number of idle instances they would like to have running. Always On was designed to allow an app to a

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-25 Thread Johan Euphrosine
On Fri, Jul 22, 2011 at 7:09 PM, Luca wrote: > Johan, > it seems to me the problem is rule 4/ below. > Shouldn't the rule be: > Idle Always On instance > Idle Dynamic instance > instead? > In this way, if you have a dynamic instance on, it will not handle traffic > unless the always-on ones are bu

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-25 Thread Johan Euphrosine
Hi, I will ask to the engineer team and get back to you in thread. Hope that helps. On Fri, Jul 22, 2011 at 5:33 PM, Tom Phillips wrote: > When are threaded instances considered "busy"? > > If it is while they are serving only a one request it would explain > why Always on instances for threade

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-25 Thread Johan Euphrosine
On Fri, Jul 22, 2011 at 8:57 PM, Galoch wrote: > Hi Johan, > > Thanks for the explanation. I have couple of questions on that. Thanks for showing interest in GAE internals, I'd be happy to answer those questions directly if I can, or forward them to someone who can answer them better. > 1. "1 Ho

[google-appengine] Re: GAE starting unnecessary instances

2011-07-23 Thread Francois Masurel
Geez, now even some Dynamic instances are not used too and stay idle for hours (check attachment). With the new pricing model this will get really really expensive. I hope there is definitely something going wrong. Francois -- You received this message because you are subscribed to the Google

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-22 Thread Robert Kluin
I've suspected the same thing. So far we've seen no pricing info for always on and questions about it have went unanswered. Not to mention that $2.10 / week for three always-on instances is a lot different than $40 / week. Robert On Fri, Jul 22, 2011 at 15:46, Tom Phillips wrote: > The curr

[google-appengine] Re: GAE starting unnecessary instances

2011-07-22 Thread Tom Phillips
The current behavior makes me suspect we are being prepared for Always- on being replaced completely by the new scheduler knobs. Being able to turn up the number of idle instances does make always-on somewhat redundant, as long as the idle instances stick around for a while. Also, if always-on ins

[google-appengine] Re: GAE starting unnecessary instances

2011-07-22 Thread Galoch
Hi Johan, Thanks for the explanation. I have couple of questions on that. 1. "1 Hours ago while all your Always On instance were busy and you had a burst of incoming requests" While this may be true when my Always On instances were "busy" running some stuff but what about when 2 Always On instanc

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-22 Thread Luca
Johan, it seems to me the problem is rule 4/ below. Shouldn't the rule be: Idle Always On instance > Idle Dynamic instance instead? In this way, if you have a dynamic instance on, it will not handle traffic unless the always-on ones are busy. So when the traffic decreases, and the dynamic

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-22 Thread Rob Coops
1/ Idle Always On instance > Spawning a new Dynamic instance 2/ Spawning a new Dynamic instance > Busy Always On instance 3/ Idle Dynamic instance > Busy Always On instance 4/ Idle Dynamic instance > Idle Always On instance So App engine prefers to use bored Always On instances over spawning new d

[google-appengine] Re: GAE starting unnecessary instances

2011-07-22 Thread Tom Phillips
When are threaded instances considered "busy"? If it is while they are serving only a one request it would explain why Always on instances for threaded Java are now severely under- utilized. /Tom On Jul 22, 10:57 am, Johan Euphrosine wrote: > HI Galoch, > > Thanks for the followup, > > I thin

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-22 Thread Johan Euphrosine
HI Galoch, Thanks for the followup, I think you are experiencing a combinaison fo the two following rules I was pointing to in my previous email: (> reads as has priority for handling the incoming request) 2/ Spawning a new Dynamic instance > Busy Always On instance 4/ Idle Dynamic instance > Idl

[google-appengine] Re: GAE starting unnecessary instances

2011-07-22 Thread Francois Masurel
Still very close from Galoch situation, mine is a bit different. My Dynamic instances dont seem to be recycled but still my Resident instances dont seem to be used at all (see attachment). Why ? Francois -- You received this message because you are subscribed to the Google Groups "Google App

[google-appengine] Re: GAE starting unnecessary instances

2011-07-22 Thread Galoch
@Johan, The issue is not about Always On instance being busy. Its actually the other way ... the Always On instance is never busy ... at least that is what we observed in last 3-4 days. Your explanation may be partly true since this behavior keeps on changing. For e.g. I have a snapshot of instanc

Re: [google-appengine] Re: GAE starting unnecessary instances

2011-07-21 Thread Johan Euphrosine
After speaking with Engs, I think I can explain what is going on: Here are the current scheduling rules: (> reads as has priority for handling the incoming request) 1/ Idle Always On instance > Spawning a new Dynamic instance 2/ Spawning a new Dynamic instance > Busy Always On instance 3/ Idle Dy

[google-appengine] Re: GAE starting unnecessary instances

2011-07-19 Thread Galoch
Same here. Seems like GAE is totally ignoring Always On instances. I also noticed that even with no user hitting our app and a single cron job that runs every 5 minutes it is still spinning instances every 3 minutes and then killing them in 2 minutes. This has been happening since after the upgrad

[google-appengine] Re: GAE starting unnecessary instances

2011-07-19 Thread Francois Masurel
Resident instances don't seem to be used at all (check attachment). It's probably a bug on GAE side. Any googlers around ? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com

[google-appengine] Re: GAE starting unnecessary instances

2011-07-19 Thread tempy
I'm still seeing this too... On Jul 19, 1:39 pm, Tom Phillips wrote: > http://code.google.com/status/appengineshows they are investigating > issue(s) with Java currently.. > > On Jul 19, 7:28 am, Tom Phillips wrote: > > > > > > > > > Yes, seeing the same problem on all my apps. Java, M/S. > > >

[google-appengine] Re: GAE starting unnecessary instances

2011-07-19 Thread Tom Phillips
http://code.google.com/status/appengine shows they are investigating issue(s) with Java currently.. On Jul 19, 7:28 am, Tom Phillips wrote: > Yes, seeing the same problem on all my apps. Java, M/S. > > Raise a production issue Francois? > > /Tom > > On Jul 19, 6:34 am, Francois Masurel wrote: >

[google-appengine] Re: GAE starting unnecessary instances

2011-07-19 Thread Tom Phillips
Yes, seeing the same problem on all my apps. Java, M/S. Raise a production issue Francois? /Tom On Jul 19, 6:34 am, Francois Masurel wrote: > Even 3 now, for just just a few new requests. Geez. > > It's getting really expensive :-( > >  Instances-VinoCities_3.jpg > 43KViewDownload -- You rece

[google-appengine] Re: GAE starting unnecessary instances

2011-07-19 Thread Francois Masurel
Even 3 now, for just just a few new requests. Geez. It's getting really expensive :-( -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/xZG_ZE6yDVEJ

[google-appengine] Re: GAE starting unnecessary instances

2011-07-19 Thread Francois Masurel
It looks like 2 resident instances and 1 dynamic are stalled as they didn't serve any request for the last minute (check attachment). But still one dynamic instance was started less than 2 minutes ago. Is it normal ? -- You received this message because you are subscribed to the Google Groups