Re: [appengine-java] Re: Why should app startup times be a problem.
Hi Jeff, In other words, Twig cannot perform the simple query: IterableObject foo = ofy.query().ancestor(yourobject); If you ever want to support something like this, you will need a registration process. If you ever want to support true polymorphic queries, you will need a registration process. Slim3 supports kindless ancestor queries and true polymorphic queries, and does not need a registration process. http://sites.google.com/site/slim3appengine/slim3-datastore/queries-and-indexes/introducing-queries http://sites.google.com/site/slim3appengine/slim3-datastore/polymorphic-model Yasuo Higa -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
On 1 Apr 2010, at 12:51, Jeff Schnitzer wrote: In other words, Twig cannot perform the simple query: IterableObject foo = ofy.query().ancestor(yourobject); All queries that are possible with the low-level datastore are possible with Twig because there are low-level to type-safe bridge methods. So you can simply do a typeless low-level ancestor query and then get Twig to convert the resulting Entities into typesafe instances. Quite frankly there just hasn't yet been the need to build un-typed queries into the API. But if a good common use case arises then it would fit in nicely to the FindCommand. If you ever want to support something like this, you will need a registration process. That is a completely incorrect conclusion to jump to. As I said, there actually was a very similar untyped query in the API that I removed because it didn't seem so useful. I am curious about what limitations Objectify has that makes this a requirement. This thread (or this forum) is probably not the best place to discuss the internal implementation details of future features for Twig or Objectify so it might be better to take this topic offline. If you ever want to support true polymorphic queries, you will need a registration process. Just wrong. What ever makes you claim that? Don't claim this short-sightedness as a feature. Think twice before posting replies at the end of a long day. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
On Thu, Apr 1, 2010 at 12:45 AM, John Patterson jdpatter...@gmail.com wrote: All queries that are possible with the low-level datastore are possible with Twig because there are low-level to type-safe bridge methods. So you can simply do a typeless low-level ancestor query and then get Twig to convert the resulting Entities into typesafe instances. That's nonsense. This could only work if you've either encoded the full java package and classname into the Entity (a colossal mistake) or you've somehow registered a mapping (typically from Kind, but it could be any synthetic field) to the java class ahead of time. This means that Your framework can't perform magic. A truly polymorphic query (of which ancestor queries are one flavor) doesn't know the concrete type of the object until it's fetched from the datastore. If you don't understand this, you haven't thought about the problem enough. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
On 1 Apr 2010, at 15:04, Jeff Schnitzer wrote: or you've somehow registered a mapping (typically from Kind, but it could be any synthetic field) to the java class ahead of time. This means that You are making a classic premature optimization mistake. Twig is built on the principal that it should work out of the box with *zero* configuration and then allow optimizations to increase performance after the application is up and running. According to this principal safe defaults are needed for many settings: all fields are indexed by default, all fields are stored by default and all references are stored as independent entities (c.f. embedded, parent or child entities). The only String value that can uniquely identify a class with no registration is its fully qualified type name. I see no problem using that kind name as a sensible default. There is nothing special about it - its just a default that can be changed later. In fact I recommend defining a shorter name as an optimisation to save datastore space. If you rename your class and want to keep the same data you will need to explicitly define the kind name for the class. Your framework can't perform magic. Actually it does contain a small bit of magic but it's not well documented. A truly polymorphic query (of which ancestor queries are one flavor) doesn't know the concrete type of the object until it's fetched from the datastore. If you don't understand this, you haven't thought about the problem enough. Chill Jeff... I've thought about the problem and so has Yasuo. I imagine this sensible default will eventually make its way into Objectify once you think about it some more :) Now lets keep this in perspective: on my mac mini I can run about 3 million reflective calls to get a constructor in a second. I do not claim that enforced registration is bad for startup times - it is almost insignificant. Its just a pain in the ass when a default can handle it for you out-of-the-box. John -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
On 1 Apr 2010, at 14:58, Jeff Schnitzer wrote: If you go with #2, you've just created a framework that will break your datastore when you rename your Java objects. I'll assume nobody thinks this is actually a good idea. From a quick browse of the Objectify source code it looks to me like the default is to use Class.getSimpleName() for the kind name which will also break on renaming and doesn't even have the advantage of working out-of-the-box with no registration. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
On Thu, Apr 1, 2010 at 1:57 AM, John Patterson jdpatter...@gmail.com wrote: You are making a classic premature optimization mistake. [...] The only String value that can uniquely identify a class with no registration is its fully qualified type name. I see no problem using that kind name as a sensible default. There is nothing special about it - its just a default that can be changed later. In fact I recommend defining a shorter name as an optimisation to save datastore space. If you rename your class and want to keep the same data you will need to explicitly define the kind name for the class. 1) I vehemently disagree that fully-qualified java class names should show up in the datastore. It's one of those things that nobody notices at the beginning and then becomes cruft that is difficult to change sometime later. Ask any DBA what they think of creating tables named com.mycompany.project.component.entity.MyEntity - and it's *way* easier to change table names on an RDBMS than it is in GAE. 2) As you note, long kind names will significantly increase your storage requirements. Full package paths in the kind name are just a bad idea. 3) If you rename your class (or use a shorter name), you will need to register your entities ahead of time. That is, if you want polymorphic queries to work. So... you've avoided registration in the simple, un-shortened case (which you don't recommend) by storing full java package paths in the datastore. You call registration premature optimization, I call it planning ahead. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
On Thu, Apr 1, 2010 at 2:19 AM, John Patterson jdpatter...@gmail.com wrote: From a quick browse of the Objectify source code it looks to me like the default is to use Class.getSimpleName() for the kind name which will also break on renaming and doesn't even have the advantage of working out-of-the-box with no registration. This approach has quite a few advantages: * It tends to be short * You can repackage at will * Even changing the classname is merely a question of adding @Entity(name=OldName) If Objectify users always register their classes explicitly, their code will always work. Not most of the time or for most queries but *all of the time*. It would be trivial to autoregister classes on first explicit use, but it wouldn't work *all of the time*. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
On 1 Apr 2010, at 16:20, Jeff Schnitzer wrote: 1) I vehemently disagree that fully-qualified java class names should show up in the datastore. It's one of those things that nobody notices at the beginning and then becomes cruft that is difficult to change sometime later. Ask any DBA what they think of creating tables named com.mycompany.project.component.entity.MyEntity - and it's *way* easier to change table names on an RDBMS than it is in GAE. One string is pretty much as good as another as long as it is unique for a class. Whether you like that string is of secondary importance to supporting the goal of zero-configuration 2) As you note, long kind names will significantly increase your storage requirements. Full package paths in the kind name are just a bad idea. They are un-optimised. Optimise them later when your app or prototype is running and its time to tune performance. Personally, about half of my data models explicitly define their kind name. The rest either are not present in such large numbers to impact storage or have not been renamed. 3) If you rename your class (or use a shorter name), you will need to register your entities ahead of time. That is, if you want polymorphic queries to work. Yes. That is assuming you want to keep the data that was stored with the old kind name. During development/prototyping this is not always the case. So... you've avoided registration in the simple, un-shortened case (which you don't recommend) Exactly. The simple case is not optimised. You call registration premature optimization, I call it planning ahead. Different goals. Working out-of-the-box with no configuration is one of the basic features of the using the ObjectDatastore. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
On 1 Apr 2010, at 16:26, Jeff Schnitzer wrote: On Thu, Apr 1, 2010 at 2:19 AM, John Patterson jdpatter...@gmail.com wrote: From a quick browse of the Objectify source code it looks to me like the default is to use Class.getSimpleName() for the kind name which will also break on renaming and doesn't even have the advantage of working out-of-the-box with no registration. This approach has quite a few advantages: * It tends to be short * You can repackage at will * Even changing the classname is merely a question of adding @Entity(name=OldName) If Objectify users always register their classes explicitly, their code will always work. Not most of the time or for most queries but *all of the time*. It would be trivial to autoregister classes on first explicit use, but it wouldn't work *all of the time*. These advantages are also present with Twig (and presumably Slims implementation?). The difference is one of defaults - you do not *have* to register classes to get started with Twigs ObjectDatastore. The defaults will just work. To be clear, Twig *never* registers kind names for Classes on first use. What it does do is cache reflective meta-data on first use as an optimisation. But admittedly, this is hardly even relevant in the context of an RPC call. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
Hi: On Wed, 2010-03-31 at 22:51 -0700, Jeff Schnitzer wrote: On Wed, Mar 31, 2010 at 6:41 PM, jd jdpatter...@gmail.com wrote: On Apr 1, 3:14 am, Jeff Schnitzer j...@infohazard.org wrote: What does Twig do when someone issues a type-less query? datastore.find().addFilter(color, EQUAL, green). returnResultsNow() The expression above is actually not possible to enter using the fluent commands. You can only set a filter on the interface that is returned from calling FindCommand.type(Class?) It is also illegal according to the datastore docs for a Query with no kind: Currently the only operations supported on a kind-less query are filter by __key__, ancestor, and order by __key__ ascending In other words, Twig cannot perform the simple query: IterableObject foo = ofy.query().ancestor(yourobject); This is the required pattern shown in Google IO scalable application for GAE. Duong BaTien DBGROUPS and BudhNet If you ever want to support something like this, you will need a registration process. If you ever want to support true polymorphic queries, you will need a registration process. Don't claim this short-sightedness as a feature. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
On 1 Apr 2010, at 19:02, Duong BaTien wrote: Hi: IterableObject foo = ofy.query().ancestor(yourobject); This is the required pattern shown in Google IO scalable application for GAE. If you are talking about the million fan out problem in Bret Slatkins talk that was also a typed query. So more like this: IterableObject foo = ofy.query(MessageIndex.class).ancestor(message); -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Why should app startup times be a problem.
David, Ikai: Thanks very much for those blog posts on reducing the cold start time. I think 1 to 2 seconds is reasonable. I'm definitely going to take a closer look at Objectify. ++Steve On Mar 30, 4:54 pm, Ikai L (Google) ika...@google.com wrote: David, that post mirrors many of the points made here: http://www.answercow.com/2010/03/google-app-engine-cold-start-guide-f... There's one or two more tips on that page. On Tue, Mar 30, 2010 at 12:47 PM, David Chandler turboman...@gmail.comwrote: In the mean time, here are some ideas for reducing startup times by shrinking our apps. I went from 8.1s to 2.5s mainly by eliminating Guice, and I would expect similar results with Spring. I can definitely live with 2.5s... http://turbomanage.wordpress.com/2010/03/26/appengine-cold-starts-con... /dmc On Mar 30, 3:04 pm, Baz b...@thinkloop.com wrote: Great information, Ikai. I really feel that instances should be completely avoided in concept and language on the GAE. What if the feature was simply an enable/disable deal called Warm Scale. If it were enabled, then your *next* instance would always be warm, regardless of how many instances you already had. This would be most noticeable and suitable for low QPS production apps that are constantly going from 0 to 1 instances (as you mentioned), but it could still be important for others, say, for a super-high-profile site, or a situation where your QPS is right at the threshold of instances and oscillating back and forth between two instances. Whatever the situation, if the solution were generalized like that, and most importantly not tied to a SPECIFIC NUMBER of instances, it would be up to the user to decide how important it was for them and whether to enable it. Cheers, Baz -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
Yeah, that blog post was awesome. I started moving to Twig last night. I understand and appreciate the concerns of others that frameworks are our friends, and that it's not unreasonable to use them, but at the same time, CPU and network optimization are even more important, so long as the barrier to achieve them isn't too high. From the few hours I've spent with Twig, it seems easier - not harder than JDO for everything I'm doing. If you want to use JPA or JDO because you're already familiar with them, that's great and all, but the mapping to BigTable isn't that straight forward, so cramming the framework on top might not be the answer. Create a data store tier, wrap all Twig/Objectify logic in there, and while your app won't be as portable as if you were using JDO, it'll perform better, use less resources, and not be terribly dificult to port later on, if you have to. My $0.02 --- Sent from my iPad On Mar 31, 2010, at 6:23 AM, SRF srfar...@gmail.com wrote: David, Ikai: Thanks very much for those blog posts on reducing the cold start time. I think 1 to 2 seconds is reasonable. I'm definitely going to take a closer look at Objectify. ++Steve On Mar 30, 4:54 pm, Ikai L (Google) ika...@google.com wrote: David, that post mirrors many of the points made here: http://www.answercow.com/2010/03/google-app-engine-cold-start-guide- f... There's one or two more tips on that page. On Tue, Mar 30, 2010 at 12:47 PM, David Chandler turboman...@gmail.comwrote: In the mean time, here are some ideas for reducing startup times by shrinking our apps. I went from 8.1s to 2.5s mainly by eliminating Guice, and I would expect similar results with Spring. I can definitely live with 2.5s... http://turbomanage.wordpress.com/2010/03/26/appengine-cold-starts-con ... /dmc On Mar 30, 3:04 pm, Baz b...@thinkloop.com wrote: Great information, Ikai. I really feel that instances should be completely avoided in concept and language on the GAE. What if the feature was simply an enable/ disable deal called Warm Scale. If it were enabled, then your *next* instance would always be warm, regardless of how many instances you already had. This would be most noticeable and suitable for low QPS production apps that are constantly going from 0 to 1 instances (as you mentioned), but it could still be important for others, say, for a super-high-profile site, or a situation where your QPS is right at the threshold of instances and oscillating back and forth between two instances. Whatever the situation, if the solution were generalized like that, and most importantly not tied to a SPECIFIC NUMBER of instances, it would be up to the user to decide how important it was for them and whether to enable it. Cheers, Baz -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle- appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Enginehttp:// googleappengine.blogspot.com|http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
On 31 Mar 2010, at 17:58, Blake Caldwell wrote: From the few hours I've spent with Twig, it seems easier - not harder than JDO for everything I'm doing. Nice to hear. You should get improved in ease of use from all of the non-standard persistence interfaces designed specifically at the datastore - just choose the features you need. Twigs ObjectDatastore aims to be as efficient as hand coded low-level solutions with no startup overhead. Unlike some of the other frameworks there is no registration process for your data model classes - each class is analyzed the first time it is used per server instance and the meta- data is cached for the duration of the instance. One user found that queries which had to be run as tasks previously with JDO could now be calculated on the fly in real time by using the ObjectDatastore. Performance can be compared directly to JDO - I believe that Twig is the only option which allows you to reuse you stored JDO data - correct me if I am wrong on that. If you want to use JPA or JDO because you're already familiar with them, that's great and all, but the mapping to BigTable isn't that straight forward, so cramming the framework on top might not be the answer. Create a data store tier, wrap all Twig/Objectify logic in there, and while your app won't be as portable as if you were using JDO, it'll perform better, use less resources, and not be terribly dificult to port later on, if you have to. Exactly. If you ever find yourself creating DTO's to abstract away details of the persistence framework you use then consider that data models in Twig *are* your DTOs - they have no dependencies at all. Pure POJOs which are really simple to port to any persistence interface you choose. It also makes working with GWT a breeze because you can send a whole object graph over the wire without having to fetch each item by Key (or KeyT) My $0.02 $0.04 total -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
looks interesting. I think I will give this a try. Can someone give any advice on doing full-text search over datastore entities? Compass+lucene is by far the heaviest part of my application. On Wed, Mar 31, 2010 at 3:36 PM, John Patterson jdpatter...@gmail.comwrote: On 31 Mar 2010, at 17:58, Blake Caldwell wrote: From the few hours I've spent with Twig, it seems easier - not harder than JDO for everything I'm doing. Nice to hear. You should get improved in ease of use from all of the non-standard persistence interfaces designed specifically at the datastore - just choose the features you need. Twigs ObjectDatastore aims to be as efficient as hand coded low-level solutions with no startup overhead. Unlike some of the other frameworks there is no registration process for your data model classes - each class is analyzed the first time it is used per server instance and the meta-data is cached for the duration of the instance. One user found that queries which had to be run as tasks previously with JDO could now be calculated on the fly in real time by using the ObjectDatastore. Performance can be compared directly to JDO - I believe that Twig is the only option which allows you to reuse you stored JDO data - correct me if I am wrong on that. If you want to use JPA or JDO because you're already familiar with them, that's great and all, but the mapping to BigTable isn't that straight forward, so cramming the framework on top might not be the answer. Create a data store tier, wrap all Twig/Objectify logic in there, and while your app won't be as portable as if you were using JDO, it'll perform better, use less resources, and not be terribly dificult to port later on, if you have to. Exactly. If you ever find yourself creating DTO's to abstract away details of the persistence framework you use then consider that data models in Twig *are* your DTOs - they have no dependencies at all. Pure POJOs which are really simple to port to any persistence interface you choose. It also makes working with GWT a breeze because you can send a whole object graph over the wire without having to fetch each item by Key (or KeyT) My $0.02 $0.04 total -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Why should app startup times be a problem.
Here is some more insight. http://bit.ly/d2lOnv Looks like I was way off on numbers for Grails. Like I said, We should think less about work arounds, and frameworks-on-diet (not that they dont work) and rather look at an out of box solution. -Sarath. On Mar 31, 2:29 am, Sarath sarath...@yahoo.com wrote: This is getting really interesting. I have been follwoing this post from a couple of days. I managed to see What exactly is ammounting to delayed startup. Some have suggested to remove spring/di/jpa/jdo. While this is true, it solves much of the problem - These are the reason why I would choose java over python. I may be wrong here - But I tried this experiment. I had four versions of a simple java app. one with grails one with spring (full di - context: scan etc) one with spring (lazyloading, ) one with slim1.0 one with NOTHING (only commons logging) I realised there are two parts to the load time. The load time from google. The load time from the libraries. I believe from a developers perspective the second one should NOT be considered for optimization. For #1 a recommendation is warm startup. While in theory it might sound good, the elasticity and managment will take it back to EC2 type like baz suggests. For my experiment the results are like this Version - Google Warmup - Framework Warmup with grails - ~9+ secs - ~4secs with spring (full di - context: scan etc) - ~4+ secs - ~3 secs with spring (lazyloading, ) - ~4 secs - ~2 secs with slim1.0 - ~2-3 secs - 1 secs with NOTHING (only commons logging) - ~2 secs - 1 sec Now I dont know what exactly Google spins down - I assume just the apps - not the whole container. In which case, There is a lot of load while loading the libraries. I can only guess, (corrections expected from technical team at google) most of the time take should be during the class loading. Grails/Spring have a bunch of libraries that are almost 1 mb some times. An assumable solution would be to preload / share these libraries in the parent classloader and have some kind of manifest either in app- web.xml or manifest.mf to use them. It may not be a fully loaded osgi type of a stack but a noval hierachied classloading with ways to selectively load libraries. Otherwise there be some other classloading mechanism that is fast. like probably from memcache. Google tech team - to comment/ I will try to upload some actual numbers from my logs and post the information. -Sarath On Mar 31, 1:54 am, Ikai L (Google) ika...@google.com wrote: David, that post mirrors many of the points made here: http://www.answercow.com/2010/03/google-app-engine-cold-start-guide-f... There's one or two more tips on that page. On Tue, Mar 30, 2010 at 12:47 PM, David Chandler turboman...@gmail.comwrote: In the mean time, here are some ideas for reducing startup times by shrinking our apps. I went from 8.1s to 2.5s mainly by eliminating Guice, and I would expect similar results with Spring. I can definitely live with 2.5s... http://turbomanage.wordpress.com/2010/03/26/appengine-cold-starts-con... /dmc On Mar 30, 3:04 pm, Baz b...@thinkloop.com wrote: Great information, Ikai. I really feel that instances should be completely avoided in concept and language on the GAE. What if the feature was simply an enable/disable deal called Warm Scale. If it were enabled, then your *next* instance would always be warm, regardless of how many instances you already had. This would be most noticeable and suitable for low QPS production apps that are constantly going from 0 to 1 instances (as you mentioned), but it could still be important for others, say, for a super-high-profile site, or a situation where your QPS is right at the threshold of instances and oscillating back and forth between two instances. Whatever the situation, if the solution were generalized like that, and most importantly not tied to a SPECIFIC NUMBER of instances, it would be up to the user to decide how important it was for them and whether to enable it. Cheers, Baz -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at
[appengine-java] Re: Why should app startup times be a problem.
Blake, The versions I have are primarily bare. And only used to measure bare minimum *cold start up time*. The benchmark is ONLY for startup-time measurement. And to show that the more (and heavy) libraries you add, The more is the cold startup time. And THAT could be delt with other technical solutions than to keep instances warm. That way all container resources will be consumed sooner or later. Hence the very few Java/Jee hosting providers. I understand some code optimization need to be done on the application to get better performance. No disagreement on that.My post only goes to counter the aspect of keeping *warm instances*. As pointed out by others, it just is against the concept of *cloud*. -Sarath. On Mar 31, 8:05 pm, Blake blakecaldw...@gmail.com wrote: Cool post Sarath, thanks for sharing. I'm also scrambling to get better performance, but I'm getting nowhere near as high CPU times as you are. What types of queries are you doing? I've restructured my app so that all of my queries are by keys. I've denormalized, serialized child objects into parents when I didn't need to query the children. I'm not going to assume I know what your app's doing, and starting handing out advice on ignorance, but we all need to accept that this isn't a traditional relational data store. We need to do more with our write operations in preparation for specific read operations. Trade-offs need to be made with every request. Even with 100 views per day, our app is basically in a system that's getting pummeled, so we need to code as if we're getting 1million pageviews per day. There's all kinds of tricks you can do (again, assuming you're not doing them already). If the CSS is dynamically generated, cache it and/or store it as a text field in a table. Use AJAX to give the perception of a quick-loading app by putting as much into the html files as you can, with spinners on the client as the data is loaded asynchronously. That extra work will even save you money once you get above the free quota. What are you doing in those requests that can't be cleaned up for better App Engine performance? - a side note to this thread, not necessarily Sarath - I agree with everyone on what would be nice. I'd love to make a fully relational database that handles ad hoc queries and no awareness of the underlying framework, but the system is what it is, so let's embrace it and squeeze every cpu clock cycle and millisecond of wait time out of it. If we can't do that, then why don't we just move to LAMP hosts for $10/month? On Mar 31, 10:19 am, Sarath sarath...@yahoo.com wrote: Here is some more insight.http://bit.ly/d2lOnv Looks like I was way off on numbers for Grails. Like I said, We should think less about work arounds, and frameworks-on-diet (not that they dont work) and rather look at an out of box solution. -Sarath. On Mar 31, 2:29 am, Sarath sarath...@yahoo.com wrote: This is getting really interesting. I have been follwoing this post from a couple of days. I managed to see What exactly is ammounting to delayed startup. Some have suggested to remove spring/di/jpa/jdo. While this is true, it solves much of the problem - These are the reason why I would choose java over python. I may be wrong here - But I tried this experiment. I had four versions of a simple java app. one with grails one with spring (full di - context: scan etc) one with spring (lazyloading, ) one with slim1.0 one with NOTHING (only commons logging) I realised there are two parts to the load time. The load time from google. The load time from the libraries. I believe from a developers perspective the second one should NOT be considered for optimization. For #1 a recommendation is warm startup. While in theory it might sound good, the elasticity and managment will take it back to EC2 type like baz suggests. For my experiment the results are like this Version - Google Warmup - Framework Warmup with grails - ~9+ secs - ~4secs with spring (full di - context: scan etc) - ~4+ secs - ~3 secs with spring (lazyloading, ) - ~4 secs - ~2 secs with slim1.0 - ~2-3 secs - 1 secs with NOTHING (only commons logging) - ~2 secs - 1 sec Now I dont know what exactly Google spins down - I assume just the apps - not the whole container. In which case, There is a lot of load while loading the libraries. I can only guess, (corrections expected from technical team at google) most of the time take should be during the class loading.
Re: [appengine-java] Re: Why should app startup times be a problem.
On Wed, Mar 31, 2010 at 6:36 AM, John Patterson jdpatter...@gmail.com wrote: Unlike some of the other frameworks there is no registration process for your data model classes - each class is analyzed the first time it is used per server instance and the meta-data is cached for the duration of the instance. What does Twig do when someone issues a type-less query? datastore.find().addFilter(color, EQUAL, green). returnResultsNow() Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
On Wed, Mar 31, 2010 at 6:41 PM, jd jdpatter...@gmail.com wrote: On Apr 1, 3:14 am, Jeff Schnitzer j...@infohazard.org wrote: What does Twig do when someone issues a type-less query? datastore.find().addFilter(color, EQUAL, green). returnResultsNow() The expression above is actually not possible to enter using the fluent commands. You can only set a filter on the interface that is returned from calling FindCommand.type(Class?) It is also illegal according to the datastore docs for a Query with no kind: Currently the only operations supported on a kind-less query are filter by __key__, ancestor, and order by __key__ ascending In other words, Twig cannot perform the simple query: IterableObject foo = ofy.query().ancestor(yourobject); If you ever want to support something like this, you will need a registration process. If you ever want to support true polymorphic queries, you will need a registration process. Don't claim this short-sightedness as a feature. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
Well, both Baz and Shawn are correct. Shawn is correct from the economical point of view. Baz is correct from the techinical point of view (no sane user would wait 15 seconds for a page to appear, right?). The great thing is that both can be done at the same time. Warmed instances can be brought up in the background without the user knowing about it. Cheers, Guillermo. On Mon, Mar 29, 2010 at 9:01 PM, Shawn Brown big.coffee.lo...@gmail.comwrote: It's boggled my mind from day 1 why instances aren't loaded in the background. I always assumed it would be addressed What is difficult to understand? 1 server has X resources warmed instanced require Y resources X - YN = resources left over for the server to fulfill requests. By reducing N (the number of warmed instances), you can increase the capacity to fulfill actual requests. By allowing us to pay for our own N, Google can presumably increase the capacity of each server or can increase the number of servers. I can see how it can cost Google to have unused apps loaded and ready to immediately serve. Can't you. I don't see what the difficulty understanding is. Shawn -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Saludos cordiales, Guillermo Schwarz Sun Certified Enterprise Architect -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
Shawn, By the same token google could cut all their hardware in half and double page loads, check out this formula: NumberOfRequests / HardwareQuality = PageLoad Therefore, Google should power the entire GAE infrastructure with a netbook - yes each page would take 2.5 decades to load, but imagine the cost savings! The main point of my post is the model, not the money. I am more than willing to pay for AppEngine as I love it and think it's the future. The issue is that I don't want to pay and manage instances because in my opinion that goes against the model. I would prefer to pay a higher rate (or the equivalent money some other way) without the knowledge of instances. With all that said, and re-reading some other posts, I may actually be wrong here... To be sure, if I pay for a warmed instance and, say, my site gradually becomes more and more popular and gradually goes from 1 instance to 11 - have I received the benefit of a warmed instance at every step of the way? Is my twelfth instance now the warm one? Cheers, Baz -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
We're still working through the details of reserved instances. The constructive feedback on this thread is useful; it is theoretically possible to pay for an unused instance that requests spill into when they go over capacity of the current spun up requests. That is - pay for one more instance than you need to avoid a loading request - for many people, they would go from 0-1, and for some folks, they would go from 1-2, and so forth. Even if this isn't the case when we first launch the feature (remember that we are always iterating to improve App Engine), the issue with many of our developers looking for this feature is that they are relatively low QPS applications that experience loading requests on every single request. As a result, a high percentage of their requests take several seconds, and these are the developers we intend on serving with this feature. At the QPS tier where you spin up additional instances to handle load, it is a much lower % - an almost negligible % of requests that take several seconds. While this is far from ideal, we've heard very little feedback from our developers running high QPS (100s - 1000s QPS) applications about this. I suspect it simply has to do with what users have expected with regards to latency spikes with the internet in general and just pass it off. Heck, I refresh pages several times at home sometimes, but that's because my DSL connection is terrible ... and probably also related to the fact that my WiFi router is next to a microwave. I'm not defending it, as in our perfect scenario all requests are served in sub 200ms. Businesses may or may not be willing to pay the cost to deal with these requests. If any of you are running high QPS applications, have these additional loading requests affected your business? If these requests have caused measurable loss of revenue or usage, then you'd have to weigh this as well as its opportunity cost against the (still undecided) cost of keeping serving instances ready. On Tue, Mar 30, 2010 at 7:57 AM, Baz b...@thinkloop.com wrote: Shawn, By the same token google could cut all their hardware in half and double page loads, check out this formula: NumberOfRequests / HardwareQuality = PageLoad Therefore, Google should power the entire GAE infrastructure with a netbook - yes each page would take 2.5 decades to load, but imagine the cost savings! The main point of my post is the model, not the money. I am more than willing to pay for AppEngine as I love it and think it's the future. The issue is that I don't want to pay and manage instances because in my opinion that goes against the model. I would prefer to pay a higher rate (or the equivalent money some other way) without the knowledge of instances. With all that said, and re-reading some other posts, I may actually be wrong here... To be sure, if I pay for a warmed instance and, say, my site gradually becomes more and more popular and gradually goes from 1 instance to 11 - have I received the benefit of a warmed instance at every step of the way? Is my twelfth instance now the warm one? Cheers, Baz -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
Great information, Ikai. I really feel that instances should be completely avoided in concept and language on the GAE. What if the feature was simply an enable/disable deal called Warm Scale. If it were enabled, then your *next* instance would always be warm, regardless of how many instances you already had. This would be most noticeable and suitable for low QPS production apps that are constantly going from 0 to 1 instances (as you mentioned), but it could still be important for others, say, for a super-high-profile site, or a situation where your QPS is right at the threshold of instances and oscillating back and forth between two instances. Whatever the situation, if the solution were generalized like that, and most importantly not tied to a SPECIFIC NUMBER of instances, it would be up to the user to decide how important it was for them and whether to enable it. Cheers, Baz -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Why should app startup times be a problem.
In the mean time, here are some ideas for reducing startup times by shrinking our apps. I went from 8.1s to 2.5s mainly by eliminating Guice, and I would expect similar results with Spring. I can definitely live with 2.5s... http://turbomanage.wordpress.com/2010/03/26/appengine-cold-starts-considered/ /dmc On Mar 30, 3:04 pm, Baz b...@thinkloop.com wrote: Great information, Ikai. I really feel that instances should be completely avoided in concept and language on the GAE. What if the feature was simply an enable/disable deal called Warm Scale. If it were enabled, then your *next* instance would always be warm, regardless of how many instances you already had. This would be most noticeable and suitable for low QPS production apps that are constantly going from 0 to 1 instances (as you mentioned), but it could still be important for others, say, for a super-high-profile site, or a situation where your QPS is right at the threshold of instances and oscillating back and forth between two instances. Whatever the situation, if the solution were generalized like that, and most importantly not tied to a SPECIFIC NUMBER of instances, it would be up to the user to decide how important it was for them and whether to enable it. Cheers, Baz -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
David, that post mirrors many of the points made here: http://www.answercow.com/2010/03/google-app-engine-cold-start-guide-for.html There's one or two more tips on that page. On Tue, Mar 30, 2010 at 12:47 PM, David Chandler turboman...@gmail.comwrote: In the mean time, here are some ideas for reducing startup times by shrinking our apps. I went from 8.1s to 2.5s mainly by eliminating Guice, and I would expect similar results with Spring. I can definitely live with 2.5s... http://turbomanage.wordpress.com/2010/03/26/appengine-cold-starts-considered/ /dmc On Mar 30, 3:04 pm, Baz b...@thinkloop.com wrote: Great information, Ikai. I really feel that instances should be completely avoided in concept and language on the GAE. What if the feature was simply an enable/disable deal called Warm Scale. If it were enabled, then your *next* instance would always be warm, regardless of how many instances you already had. This would be most noticeable and suitable for low QPS production apps that are constantly going from 0 to 1 instances (as you mentioned), but it could still be important for others, say, for a super-high-profile site, or a situation where your QPS is right at the threshold of instances and oscillating back and forth between two instances. Whatever the situation, if the solution were generalized like that, and most importantly not tied to a SPECIFIC NUMBER of instances, it would be up to the user to decide how important it was for them and whether to enable it. Cheers, Baz -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Why should app startup times be a problem.
This is getting really interesting. I have been follwoing this post from a couple of days. I managed to see What exactly is ammounting to delayed startup. Some have suggested to remove spring/di/jpa/jdo. While this is true, it solves much of the problem - These are the reason why I would choose java over python. I may be wrong here - But I tried this experiment. I had four versions of a simple java app. one with grails one with spring (full di - context: scan etc) one with spring (lazyloading, ) one with slim1.0 one with NOTHING (only commons logging) I realised there are two parts to the load time. The load time from google. The load time from the libraries. I believe from a developers perspective the second one should NOT be considered for optimization. For #1 a recommendation is warm startup. While in theory it might sound good, the elasticity and managment will take it back to EC2 type like baz suggests. For my experiment the results are like this Version - Google Warmup - Framework Warmup with grails - ~9+ secs- ~4secs with spring (full di - context: scan etc) - ~4+ secs- ~3 secs with spring (lazyloading, ) - ~4 secs - ~2 secs with slim1.0- ~2-3 secs - 1 secs with NOTHING (only commons logging) - ~2 secs - 1 sec Now I dont know what exactly Google spins down - I assume just the apps - not the whole container. In which case, There is a lot of load while loading the libraries. I can only guess, (corrections expected from technical team at google) most of the time take should be during the class loading. Grails/Spring have a bunch of libraries that are almost 1 mb some times. An assumable solution would be to preload / share these libraries in the parent classloader and have some kind of manifest either in app- web.xml or manifest.mf to use them. It may not be a fully loaded osgi type of a stack but a noval hierachied classloading with ways to selectively load libraries. Otherwise there be some other classloading mechanism that is fast. like probably from memcache. Google tech team - to comment/ I will try to upload some actual numbers from my logs and post the information. -Sarath On Mar 31, 1:54 am, Ikai L (Google) ika...@google.com wrote: David, that post mirrors many of the points made here: http://www.answercow.com/2010/03/google-app-engine-cold-start-guide-f... There's one or two more tips on that page. On Tue, Mar 30, 2010 at 12:47 PM, David Chandler turboman...@gmail.comwrote: In the mean time, here are some ideas for reducing startup times by shrinking our apps. I went from 8.1s to 2.5s mainly by eliminating Guice, and I would expect similar results with Spring. I can definitely live with 2.5s... http://turbomanage.wordpress.com/2010/03/26/appengine-cold-starts-con... /dmc On Mar 30, 3:04 pm, Baz b...@thinkloop.com wrote: Great information, Ikai. I really feel that instances should be completely avoided in concept and language on the GAE. What if the feature was simply an enable/disable deal called Warm Scale. If it were enabled, then your *next* instance would always be warm, regardless of how many instances you already had. This would be most noticeable and suitable for low QPS production apps that are constantly going from 0 to 1 instances (as you mentioned), but it could still be important for others, say, for a super-high-profile site, or a situation where your QPS is right at the threshold of instances and oscillating back and forth between two instances. Whatever the situation, if the solution were generalized like that, and most importantly not tied to a SPECIFIC NUMBER of instances, it would be up to the user to decide how important it was for them and whether to enable it. Cheers, Baz -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to
Re: [appengine-java] Re: Why should app startup times be a problem.
Hah, Blake, guess you caught us! We might be doing more stuff with that in addition to what went out with 1.3.2. On Fri, Mar 26, 2010 at 6:50 PM, Blake blakecaldw...@gmail.com wrote: Ikai - that's awesome - great to see you guys listening to us. Is that doc already outdated? Didn't you just release this? Ability to select different availability vs. latency options for Datastore On Mar 26, 1:42 pm, Ikai L (Google) ika...@google.com wrote: Keeping reserved instances has been added to our public roadmap: http://code.google.com/appengine/docs/roadmap.html http://code.google.com/appengine/docs/roadmap.htmlAs far as spinning up additional instances, there are probably a few good solutions here. We'll be best off collecting feedback when we ship reserved instances on which solution works best. On Fri, Mar 26, 2010 at 10:38 AM, gholler georgehol...@gmail.com wrote: Thanks for your replies. I think we would want an option to pay to keep at least one instance warm and ready to go at any time (I don't know what makes sense for a fee though). And as Guillermo says, that won't help us as new instances are needed to scale. There could be a fee for paying accounts to get those special startup requests, and I also don't know what makes sense for a fee there. It would also need to be able to handle more than 30 seconds, ideally it could be a task that keeps gettings called until it returns a 200-299 status. The url for the task could be specified in the appengine-web.xml. G -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Enginehttp:// googleappengine.blogspot.com|http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
JohnJ, The way I see it, you just need to pay for one warmed instance. I've measured that one instance is enough to handle 30 requests per second. Trying to execute more requests than that immediatly triggers the loading of a new instance and so far some of the requests fail. Therefore, If you are using 1 instance in real traffic (max 30 requests per second), you only need another one when you reach 31 requests per second, so that new instance in theory would all you need warmed (unless you think you will jump from one instance to 3 instance from one second to the next). Assuming all requests end before one second, and assuming the time to load an instance to be kept warmed is a few seconds, you only need 1 warmed instance added to the instances you are actually using (which should be number of requests per second / 30). Did I make myself clear? Cheers, Guillermo. On Mon, Mar 29, 2010 at 1:58 PM, Ikai L (Google) ika...@google.com wrote: Hah, Blake, guess you caught us! We might be doing more stuff with that in addition to what went out with 1.3.2. On Fri, Mar 26, 2010 at 6:50 PM, Blake blakecaldw...@gmail.com wrote: Ikai - that's awesome - great to see you guys listening to us. Is that doc already outdated? Didn't you just release this? Ability to select different availability vs. latency options for Datastore On Mar 26, 1:42 pm, Ikai L (Google) ika...@google.com wrote: Keeping reserved instances has been added to our public roadmap: http://code.google.com/appengine/docs/roadmap.html http://code.google.com/appengine/docs/roadmap.htmlAs far as spinning up additional instances, there are probably a few good solutions here. We'll be best off collecting feedback when we ship reserved instances on which solution works best. On Fri, Mar 26, 2010 at 10:38 AM, gholler georgehol...@gmail.com wrote: Thanks for your replies. I think we would want an option to pay to keep at least one instance warm and ready to go at any time (I don't know what makes sense for a fee though). And as Guillermo says, that won't help us as new instances are needed to scale. There could be a fee for paying accounts to get those special startup requests, and I also don't know what makes sense for a fee there. It would also need to be able to handle more than 30 seconds, ideally it could be a task that keeps gettings called until it returns a 200-299 status. The url for the task could be specified in the appengine-web.xml. G -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Enginehttp:// googleappengine.blogspot.com|http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Saludos cordiales, Guillermo Schwarz Sun Certified Enterprise Architect -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
Warmed instances make absolutely no sense on the GAE! It's supposed to be an invisible, infinite, platform without the notion of ram, cpu's, drives or instances - it goes completely against the heart and soul of the project. That just makes us an Amazon. It's boggled my mind from day 1 why instances aren't loaded in the background. I always assumed it would be addressed shortly - but instead we've decided to completely break the model. Why? I hate to over-simply, but it really doesn't seem like it's that difficult to make sure requests only go to ready/warm servers. I bet you could even solve 99% of the cases by simply waiting 3 minutes before serving a request. Of course, I still love the GAE :) Baz -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
It's boggled my mind from day 1 why instances aren't loaded in the background. I always assumed it would be addressed What is difficult to understand? 1 server has X resources warmed instanced require Y resources X - YN = resources left over for the server to fulfill requests. By reducing N (the number of warmed instances), you can increase the capacity to fulfill actual requests. By allowing us to pay for our own N, Google can presumably increase the capacity of each server or can increase the number of servers. I can see how it can cost Google to have unused apps loaded and ready to immediately serve. Can't you. I don't see what the difficulty understanding is. Shawn -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
