There are two schedulers already at work I believe. On-line handler and Task Queue. TQ is, I believe, optimized for longer running tasks. Google has never to my knowledge explained how these two schedulers interact managing load inside the current instance setup. I long, long ago posited that by changing TQ moderately there would be opportunities substantial system optimizations. That is a complete guess, however, knowing nothing about how the system works. Anyway, I believe TQ is the ignored, red-headed step-child within GAE. Perhaps initiated by a brilliant engineer who left, and no one wants to visit his/her code. Alternatively, it could have been a quick idea hack that everyone wants to deprecate and dispose of. Ultimately GAE today is clearly a version one. At the very least it would make sense for the OL Scheduler to analyze calls individually, but I don't think it does that based on a previous thread Brandon started about one very high latency function effecting a higher-than-necessary instance count. Hopefully the initial financials for GAE are promising, and these types of issues will get the great minds at G. applied. Seeing the many new names from Google in the forums, I'm assuming that is the case.
On Nov 24, 1:39 am, "Brandon Wirtz" <drak...@digerat.com> wrote: > I probably need more time for testing but. > > It would appear that putting slow tasks on one app and fast tasks on the > other DRASTICALLY reduces the number of required instances. > > I suspect that requests that take less than 200ms don't support concurrency, > and requests that take more than 1500 support a lot. > > So when they co-exist on the same instances the fast instances don't stack > well, and the long requests create large delays. Basically what was > requiring 8-10 instances now takes 4. Some of that may just be that my ram > usage lets me hit instance memory more often in the segmented apps because I > have less total data passing through instances. > > This makes a strong case for being able to specify Handlers to specific > instances, and having different Scheduler settings for various handlers. > > Brandon Wirtz > BlackWaterOps: President / Lead Mercenary > > Description:http://www.linkedin.com/img/signature/bg_slate_385x42.jpg > > Work: 510-992-6548 > Toll Free: 866-400-4536 > > IM: drak...@gmail.com (Google Talk) > Skype: drakegreene > > <http://www.blackwaterops.com/> BlackWater Ops > > From: google-appengine@googlegroups.com > [mailto:google-appengine@googlegroups.com] On Behalf Of Brandon Wirtz > Sent: Wednesday, November 23, 2011 5:13 PM > To: google-appengine@googlegroups.com > Subject: RE: [google-appengine] Feature Request I Think.... > > It will be a while before I'm certain, but it looks like separating long and > short requests saves me a LOT on instance $$$. I'm guessing that "volume" > and "average request time" don't get calculated well when the types of > requests change, so when site X gets crawled by Google Bot the scheduler > tunes for one setting, and why Y gets crawled it gets tune another way, and > when both get crawled at the same time all hell breaks loose. > > Latency by IP Range would be SOOOO awesome :-) those Google bots don't need > the same QoS as my Paying customers :-) > > From: google-appengine@googlegroups.com > [mailto:google-appengine@googlegroups.com] On Behalf Of Brandon Wirtz > Sent: Wednesday, November 23, 2011 4:58 PM > To: google-appengine@googlegroups.com > Subject: RE: [google-appengine] Feature Request I Think.... > > Chris, > > For my app sharing data would not be an issue. That is one of the reasons > I'm considering (and testing) using more than one App so that I can have > different Scheduler/Instance/Latency settings in each. > > In my case most of the difference is per domain, but I do know for instance > that everything in with a ?s=Keyword is a Search and will therefor take > longer than just returning some page with some data. > > The reason I was looking at "Per Handler" was that I was hoping that > something could be done where it was just instance balancing. I would have > 2 instances reserved for "Priority 0 requests" and 1 for "p1" requests. > > If P0 is busy, and p1 is not p1 serves p0 requests. > > If P0 is busy and P1 is busy more P0 instances spin up > > If P1 is busy more P1 instances spin up. > > But the "busy" state for each would be configurable. > > If P0 Request has "133 ms pending Latency set" and p1 has 1000 ms Pending > latency > > Then the waterfall would look something like this.. > > P0 Request > > Instance 1 has 166ms until free -> go next > > Instance 2 has 266ms until free -> go next > > Instance 3 is p1 handler has 106ms until free -> serve request > > P0 Request > > Instance 1 has 166ms until free -> go next > > Instance 2 has 266ms until free -> go next > > Instance 3 is p1 handler has 1206ms until free -> Create new P0 Instance > > Instance 4 is P0 -> serve request > > P1 Request > > Instance 1 is not P1 has 166ms until free -> go next > > Instance 2 is not P1 has 266ms until free -> go next > > Instance 3 is p1 handler has 106ms until free -> serve request > > P1 Request > > Instance 1 is not P1 has 166ms until free -> go next > > Instance 2 is not P1 has 266ms until free -> go next > > Instance 3 is p1 handler has 2106ms until free -> Create New p1 Instance > > Insance 4 is p1 -> serve request > > Ideally It would be nice to specify by handler, or Assigned domain. I don't > know if there is reason to have more than 2 tiers of service. For small apps > I would think 2 would do fine. I can envision large apps looking at 3, but > likely you have "Very fast", "Kind of slow" and "Should be done on the back > end". > > Where things get "interesting" is in resource balancing. The Really slow > requests being on a separate app in my app is causing extra congestion > because they have the really high Cache-Miss rate because they are > computing/gathering new data that I don't already have. So they tend to be > one off requests that don't benefit much from memcache. I'm a lazy coder so > I always check just in case because when you do get lucky and the data is > there I save > > 2000ms and that adds up. But in terms of users serve/pages served that hit > cost me a lot of the very fast page serves ability to be cached. SO. putting > my slow requests on a separate App Gives me HUGE wins because I get a better > cache hit ratio and more total Mem-cache. > > <Side bar> It doesn't appear that Memcache scales with my number of > instances, but rather is a per app limit. Where as local instance memory > goes up with my number of instances. As a result instance memory cache hits > scale up when you have a spike in traffic to 100s of page, and mem-cache > hits go down because if the number of actively requested items goes up > mem-cache stays the same size. (this also comes in to play as you start to > multi-tenant lots of websites > > <back on track> > > So for me the "non-sharing" of data across apps means I get more resources > which is a good thing, that wouldn't work for a lot of other people, who > don't have such beautifully segmented data that is happy in silo's. > Datastore not being available across apps is kind of sad. I mean I like that > if I have my PostGres or MySQL or Oracle server I can put as many apps on > the edge and talk to it as I want, Datastore in GAE doesn't really allow > that, (Though I did do some code that basically talks to DataStore via > "curl" > > I meandered, these are my thoughts "Raw" as I think things through. > > For me: > > More Apps = Harder to manage but better and more customizable performance, > no ability to have different priorities on the Same Domain > > Apps with Instance priority =Fewer resources, Easier Management, ability to > prioritize with in a single domain > > From: google-appengine@googlegroups.com > [mailto:google-appengine@googlegroups.com] On Behalf Of Chris Ramsdale > Sent: Wednesday, November 23, 2011 3:59 PM > To: google-appengine@googlegroups.com > Subject: Re: [google-appengine] Feature Request I Think.... > > Hey Brandon, > > A timely post indeed. Amongst the team, we've been discussing this exact > problem. Splitting your app out into smaller apps has a handful of benefits, > one of which is the ability to fine tune the performance settings for each > app. I'd love to hear more about your design. For example: > > - Assuming that it's required, how are you going to handle sharing data > between the two apps? > > - Are there other app-specific performance characteristics that you expect > to change? > > - Are there configuration settings that you would expect to be applied to > both apps? (E.g., a centralized list of admins or cross-app budget) > > - Any thoughts on how you would configure this with custom domains? > > -- Chris > > Product Manager, Google App Engine > > On Wed, Nov 23, 2011 at 2:27 PM, Brandon Wirtz <drak...@digerat.com> wrote: > > I am considering asking for the ability to specify Min/Max Latency in the > App.Yaml per handler. > > I am really wishing I could have Fast instances and slow instances. I have > a lot of requests that take 66ms to fill, and a lot that take 3000ms > > Concurrency seems to help some, but really I probably should be breaking > these in to two apps one for the slow stuff that it wouldn't matter if it > too 5s instead of 3s and on for the stuff that 66 or 133ms wouldn't make any > difference. > > But the ones that take 66 seem really slow when they take 3066ms and I > don't really need more instances, just more ability to arrange them. > > I had thought that by combining apps that I'd get to the point this would be > less of an issued, but since the scheduler doesn't know how long requests > are going to take it seems most inefficient at dealing with multiple "sized" > requests. > > But all my troubles would go away if "slow requests" and "fast requests" > went in to two different buckets. > > I'm willing to implement this however GAE thinks is best, so I'm posting > this as "How do you solve this? And How should I?" > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > <mailto:google-appengine%2bunsubscr...@googlegroups.com> . > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send ... > > read more » > > image001.jpg > < 1KViewDownload -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
