Hi AJ, Here's a few tips from my experience:
* If your application currently does not have much traffic, most of your requests will be "loading requests". This could explain the randomness your are seeing. The app engine needs to load and prepare your application to handle a request. In my application (java), a loading request takes around 2-6 seconds. Once your application is loaded, it will stay in memory for some time and subsequent requests will be "normal requests". See http://googleappengine.blogspot.com/2009/12/request-performance-in-java.html for more info. Also, make sure you are using the most recent version of the app engine so that pre-compilation is enabled by default. I have not tried the python implementation of the app engine yet, but I heard that it process loading requests faster. * Once my application is loaded, fetching the 20 first results of a simple query takes at most 1-2 seconds, most of the time less than a second. I use memcache to cache the derived product (usually an html page) of common requests. Once a request is cached, it will take 100-200ms to serve. * When you list the topics, make sure you are using cursors: http://code.google.com/appengine/docs/java/datastore/queriesandindexes.html#Query_Cursors * When you fetch feeds, you will need to split the work in very small chunks because of the 30 seconds limit. When you say "fails randomly", are you getting "com.google.apphosting.runtime.HardDeadlineExceededError"? From my experience, appengine behave much better when each of your tasks write to very few entities. In my application, I'm being very strict: each task writes to at most one entity. For example, I also need to parse an external feed and then update 1000 entities with the data from that feed. In order to achieve that efficiently, I spawn a thousand tasks, each one updating only one entity. Hope it helps, Marc On Feb 21, 7:02 pm, AJ Chen <ajc...@web2express.org> wrote: > I like the potential of appengine as cloud computing platform. Using > eclipse, I can code in eclipse and then one-click to deploy the changes to > production. It's awesome! It's also an exciting experience to learn the new > programing patterns GAE requires, such as task queue, object data store, > mencache. All these new stuff are fine as long as it delivers the > performance at the end. After running my apphttp://realmon9.appspot.comon > production for a while, I found the response time very often is too long, > 5-20sec, in the unusable range. Of course, the performce depends on how > complicated it is to generate the response per request. I'm going to give a > very brief description of a typical request, and appreciate your suggestion > for improving the performance.. > > My app "realmon9" <http://realmon9.appspot.com> is a social media monitoring > application designed as a component in the google cloud so that it can be > connected to enterprise CRM like salesforce. It basically allows > organization to monitor a large number of topics on social media and brings > the relevant conversations/leads to CRM for marketing/PR/support/research > purpose. The topics, twiter conversations and blogs are stored in datastore > and the operation is quite simple and straightforward. For example, listing > a list of topics or listing conversations (20 per page) for a topic. I > expect this type of viewing request takes <1 sec to respond. It requires > querying 2-4 kinds of data per request and there are only small number of > data in this initial stage. But, very often it takes 10 second to respond to > a simple request. I'm using java and JDO to query datastore. I have not > done anything to customize the index configuration yet. Where should I look > for performance optimization? > > I also use task queue to fetch feeds in the background. Because the reponse > is slow, a large percentage of simple feed fatch tasks fail randomly. > > One observation: viewing the same page (e.g. listing topics) sometimes takes > no time, but sometimes takes 10 second. it's all random, which is probably > due to the distributed nature of GAE. It may be hard to figure out what to > improve on the app side when GAE varies wildly in terms response time. > Anybody know the expected response time or behavior from GAE? > > This google app is porting from my server application > onhttp://web2express.org. I can make the responce on regular tomcat server > fast, but GAE is the uncharted torritory. I"m still learning and looking for > best practice ideas. > > thanks, > -aj > -- > AJ Chen, PhD > Chair, Semantic Web SIG, sdforum.orghttp://web2express.org > @web2express on twitter > Palo Alto, CA, USA > 650-283-4091 > *Monitoring social media in real time* -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
