[appengine-java] Re: Why should app startup times be a problem.

One of the proposed solutions here (and now officially on the roadmap - wooo!) is the ability to pay a fee to keep your JVM "warm" and reduce the currently-frequent occurrences of startup. See http://groups.google.com/group/google-appengine-java/msg/dafc502f0957c58a for an overview of this and sim

[appengine-java] Re: Why should app startup times be a problem.

Thanks for your replies. I think we would want an option to pay to keep at least one instance "warm" and ready to go at any time (I don't know what makes sense for a fee though). And as Guillermo says, that won't help us as new instances are needed to scale. There could be a fee for paying accoun

[appengine-java] Re: Why should app startup times be a problem.

Ikai, is there any other reason (aside from energy consumption) that instances will become idle after x amount of time? I've done some scripted testing and I haven't seen a constant time for when applications become idle, some blogs say 1 minute some say other values, I saw a different value. Is th

[appengine-java] Re: Why should app startup times be a problem.

Paying for warmed instances kind of defeats the notion of elastic computing, no? I have no problem paying for warm instances, I just don't think that is the solution. At some point you will eventually run out of warmed instances. I think gholler's idea holds more true to the promise of GAE. O

[appengine-java] Re: Why should app startup times be a problem.

Ikai - that's awesome - great to see you guys listening to us. Is that doc already outdated? Didn't you just release this? "Ability to select different availability vs. latency options for Datastore" On Mar 26, 1:42 pm, "Ikai L (Google)" wrote: > Keeping reserved instances has been added to ou

[appengine-java] Re: Why should app startup times be a problem.

In the mean time, here are some ideas for reducing startup times by shrinking our apps. I went from 8.1s to 2.5s mainly by eliminating Guice, and I would expect similar results with Spring. I can definitely live with 2.5s... http://turbomanage.wordpress.com/2010/03/26/appengine-cold-starts-conside

[appengine-java] Re: Why should app startup times be a problem.

This is getting really interesting. I have been follwoing this post from a couple of days. I managed to see What exactly is ammounting to delayed startup. Some have suggested to remove spring/di/jpa/jdo. While this is true, it solves much of the problem - These are the reason why I would choose j

[appengine-java] Re: Why should app startup times be a problem.

Hello, Thank you Sarath for this very nice wrap up and for the figures. I totally agree with your findings. To abandon all use of frameworks even the ones provided by GAE (e.g. JDO) would mean to give up a lot of benefits and to end up with messy code especially for large application. Also framew

[appengine-java] Re: Why should app startup times be a problem.

David, Ikai: Thanks very much for those blog posts on reducing the cold start time. I think 1 to 2 seconds is reasonable. I'm definitely going to take a closer look at Objectify. ++Steve On Mar 30, 4:54 pm, "Ikai L (Google)" wrote: > David, that post mirrors many of the points made here: > >

[appengine-java] Re: Why should app startup times be a problem.

Here is some more insight. http://bit.ly/d2lOnv Looks like I was way off on numbers for Grails. Like I said, We should think less about work arounds, and frameworks-on-diet (not that they dont work) and rather look at an out of box solution. -Sarath. On Mar 31, 2:29 am, Sarath wrote: > This is

[appengine-java] Re: Why should app startup times be a problem.

Cool post Sarath, thanks for sharing. I'm also scrambling to get better performance, but I'm getting nowhere near as high CPU times as you are. What types of queries are you doing? I've restructured my app so that all of my queries are by keys. I've denormalized, serialized child objects into p

[appengine-java] Re: Why should app startup times be a problem.

Blake, The versions I have are primarily bare. And only used to measure bare minimum *cold start up time*. The benchmark is ONLY for startup-time measurement. And to show that the more (and heavy) libraries you add, The more is the cold startup time. And THAT could be delt with other technical sol

[appengine-java] Re: Why should app startup times be a problem.

On Apr 1, 3:14 am, Jeff Schnitzer wrote: > What does Twig do when someone issues a type-less query? > > datastore.find().addFilter("color", EQUAL, "green"). returnResultsNow() The expression above is actually not possible to enter using the fluent commands. You can only set a filter on the int

Re: [appengine-java] Re: Why should app startup times be a problem.

I think what gholler means is that when a SECOND instance is being started up because of too much traffic, "in order to serve you better", what actually happens is that the client sees the startup time. Therefore keeping the first instance "warm" would serve no purpose. The way I see gholler´s pr

Re: [appengine-java] Re: Why should app startup times be a problem.

Keeping reserved instances has been added to our public roadmap: http://code.google.com/appengine/docs/roadmap.html As far as spinning up additional instances, there are probably a few good solutions here. We'll be best off collecting feedback w

Re: [appengine-java] Re: Why should app startup times be a problem.

Instances will be spun down as needed, so I'm not surprised folks are experiencing different times. On Fri, Mar 26, 2010 at 3:15 PM, luijar wrote: > Ikai, is there any other reason (aside from energy consumption) that > instances will become idle after x amount of time? I've done some > scripted

Re: [appengine-java] Re: Why should app startup times be a problem.

Hah, Blake, guess you caught us! We might be doing more stuff with that in addition to what went out with 1.3.2. On Fri, Mar 26, 2010 at 6:50 PM, Blake wrote: > Ikai - that's awesome - great to see you guys listening to us. Is > that doc already outdated? Didn't you just release this? > > "Abi

Re: [appengine-java] Re: Why should app startup times be a problem.

JohnJ, The way I see it, you just need to pay for one warmed instance. I've measured that one instance is enough to handle 30 requests per second. Trying to execute more requests than that immediatly triggers the loading of a new instance and so far some of the requests fail. Therefore, If you a

Re: [appengine-java] Re: Why should app startup times be a problem.

Warmed instances make absolutely no sense on the GAE! It's supposed to be an invisible, infinite, platform without the notion of ram, cpu's, drives or instances - it goes completely against the heart and soul of the project. That just makes us an Amazon. It's boggled my mind from day 1 why instance

Re: [appengine-java] Re: Why should app startup times be a problem.

> It's boggled my mind from day 1 why instances > aren't loaded in the background. I always assumed it would be addressed What is difficult to understand? 1 server has X resources warmed instanced require Y resources X - YN = resources left over for the server to fulfill requests. By reducing

Re: [appengine-java] Re: Why should app startup times be a problem.

Well, both Baz and Shawn are correct. Shawn is correct from the economical point of view. Baz is correct from the techinical point of view (no sane user would wait 15 seconds for a page to appear, right?). The great thing is that both can be done at the same time. Warmed instances can be brought

Re: [appengine-java] Re: Why should app startup times be a problem.

Shawn, By the same token google could cut all their hardware in half and double page loads, check out this formula: NumberOfRequests / HardwareQuality = PageLoad Therefore, Google should power the entire GAE infrastructure with a netbook - yes each page would take 2.5 decades to load, but imagin

Re: [appengine-java] Re: Why should app startup times be a problem.

We're still working through the details of reserved instances. The constructive feedback on this thread is useful; it is theoretically possible to pay for an unused instance that requests spill into when they go over capacity of the current spun up requests. That is - pay for one more instance than

Re: [appengine-java] Re: Why should app startup times be a problem.

Great information, Ikai. I really feel that "instances" should be completely avoided in concept and language on the GAE. What if the feature was simply an enable/disable deal called "Warm Scale". If it were enabled, then your *next* instance would always be warm, regardless of how many instances y

Re: [appengine-java] Re: Why should app startup times be a problem.

David, that post mirrors many of the points made here: http://www.answercow.com/2010/03/google-app-engine-cold-start-guide-for.html There's one or two more tips on that page. On Tue, Mar 30, 2010 at 12:47 PM, David Chandler wrote: > In the mean time, here are some ideas for reducing startup tim

Re: [appengine-java] Re: Why should app startup times be a problem.

Yeah, that blog post was awesome. I started moving to Twig last night. I understand and appreciate the concerns of others that frameworks are our friends, and that it's not unreasonable to use them, but at the same time, CPU and network optimization are even more important, so long as the barrier t

Re: [appengine-java] Re: Why should app startup times be a problem.

On 31 Mar 2010, at 17:58, Blake Caldwell wrote: From the few hours I've spent with Twig, it seems easier - not harder than JDO for everything I'm doing. Nice to hear. You should get improved in ease of use from all of the non-standard persistence interfaces designed specifically at the d

Re: [appengine-java] Re: Why should app startup times be a problem.

looks interesting. I think I will give this a try. Can someone give any advice on doing full-text search over datastore entities? Compass+lucene is by far the heaviest part of my application. On Wed, Mar 31, 2010 at 3:36 PM, John Patterson wrote: > > On 31 Mar 2010, at 17:58, Blake Caldwell wrote

Re: [appengine-java] Re: Why should app startup times be a problem.

On Wed, Mar 31, 2010 at 6:36 AM, John Patterson wrote: > >  Unlike some of the other frameworks there is no "registration" process for > your data model classes - each class is analyzed the first time it is used > per server instance and the meta-data is cached for the duration of the > instance.

Re: [appengine-java] Re: Why should app startup times be a problem.

On Wed, Mar 31, 2010 at 6:41 PM, jd wrote: > > On Apr 1, 3:14 am, Jeff Schnitzer wrote: >> What does Twig do when someone issues a type-less query? >> >> datastore.find().addFilter("color", EQUAL, "green"). returnResultsNow() > > The expression above is actually not possible to enter using the >

Re: [appengine-java] Re: Why should app startup times be a problem.

Hi Jeff, > In other words, Twig cannot perform the simple query: > > Iterable foo = ofy.query().ancestor(yourobject); > > If you ever want to support something like this, you will need a > registration process. > > If you ever want to support true polymorphic queries, you will need a > registratio

Re: [appengine-java] Re: Why should app startup times be a problem.

On 1 Apr 2010, at 12:51, Jeff Schnitzer wrote: In other words, Twig cannot perform the simple query: Iterable foo = ofy.query().ancestor(yourobject); All queries that are possible with the low-level datastore are possible with Twig because there are low-level to type-safe "bridge" methods

Re: [appengine-java] Re: Why should app startup times be a problem.

Let's take a step back for a moment. Fundamentally, all these API layers do is map back and forth between untyped datastore Entity objects and Java classes. Let's say the user runs a query that returns an Entity. Somehow the framework must figure out what Java class to instantiate and populate w

Re: [appengine-java] Re: Why should app startup times be a problem.

On Thu, Apr 1, 2010 at 12:45 AM, John Patterson wrote: > > All queries that are possible with the low-level datastore are possible with > Twig because there are low-level to type-safe "bridge" methods.  So you can > simply do a typeless low-level ancestor query and then get Twig to convert > the r

Re: [appengine-java] Re: Why should app startup times be a problem.

On 1 Apr 2010, at 15:04, Jeff Schnitzer wrote: or you've somehow registered a mapping (typically from Kind, but it could be any synthetic field) to the java class ahead of time. This means that You are making a classic premature optimization mistake. Twig is built on the principal that it s

Re: [appengine-java] Re: Why should app startup times be a problem.

On 1 Apr 2010, at 14:58, Jeff Schnitzer wrote: If you go with #2, you've just created a framework that will break your datastore when you rename your Java objects. I'll assume nobody thinks this is actually a good idea. From a quick browse of the Objectify source code it looks to me like th

Re: [appengine-java] Re: Why should app startup times be a problem.

On Thu, Apr 1, 2010 at 1:57 AM, John Patterson wrote: > > You are making a classic premature optimization mistake. > [...] > The only String value that can uniquely identify a class with no > "registration" is its fully qualified type name.  I see no problem using > that kind name as a sensible de

Re: [appengine-java] Re: Why should app startup times be a problem.

On Thu, Apr 1, 2010 at 2:19 AM, John Patterson wrote: > > From a quick browse of the Objectify source code it looks to me like the > default is to use Class.getSimpleName() for the kind name which will also > break on renaming and doesn't even have the advantage of working > out-of-the-box with no

Re: [appengine-java] Re: Why should app startup times be a problem.

On 1 Apr 2010, at 16:20, Jeff Schnitzer wrote: 1) I vehemently disagree that fully-qualified java class names should show up in the datastore. It's one of those things that nobody notices at the beginning and then becomes cruft that is difficult to change sometime later. Ask any DBA what they

Re: [appengine-java] Re: Why should app startup times be a problem.

On 1 Apr 2010, at 16:26, Jeff Schnitzer wrote: On Thu, Apr 1, 2010 at 2:19 AM, John Patterson wrote: From a quick browse of the Objectify source code it looks to me like the default is to use Class.getSimpleName() for the kind name which will also break on renaming and doesn't even hav

Re: [appengine-java] Re: Why should app startup times be a problem.

Hi: On Wed, 2010-03-31 at 22:51 -0700, Jeff Schnitzer wrote: > On Wed, Mar 31, 2010 at 6:41 PM, jd wrote: > > > > On Apr 1, 3:14 am, Jeff Schnitzer wrote: > >> What does Twig do when someone issues a type-less query? > >> > >> datastore.find().addFilter("color", EQUAL, "green"). returnResultsNow

Re: [appengine-java] Re: Why should app startup times be a problem.

On 1 Apr 2010, at 19:02, Duong BaTien wrote: Hi: Iterable foo = ofy.query().ancestor(yourobject); This is the required pattern shown in Google IO scalable application for GAE. If you are talking about the "million fan out" problem in Bret Slatkins talk that was also a typed query.