Hi, Some responses inline. Robert
On Tue, Jan 31, 2012 at 11:50, UseShots <goo...@useshots.com> wrote: > Hi, > > My debug logs show that imports take quite a long time. But Google > App Engine spends several seconds even before my first log entry. > The whole execution is also slowed down (even without datastore > operations). Do you see pending_ms values in the logs for these slow requests? > > During the last 10 days, I've been testing various app designs work during > the latency periods (I still see them almost every day). > > 1. The first thing I did was moved the home page request (acccountable for > ~30% of app regquests) from Django to WebApp with Django template > (different request handler files in app.yaml) > > The WebApp w/Django requests took roughly 3 times as little to execute as > normal Django requests during the high latency periods. This still meant > 50-100 seconds per loading request. Well, at least it is faster. Could the remaining delay be due to imports in other portions of the app? In other words, once you start importing some portion of your app, does the entire thing wind up needing to be loaded? > > 2. I moved another frequent request (also ~30% of all requests) to a new > request handler in a separate file. This time it was WebApp without Django > templates. This request just calculates an MD5 hash and sets a cookie. Very > simple. > > During the high latency periods it took about 25 seconds to load. Faster > than requests with Django dependencies, but still too slow. > > 3. Converted the WebApp request handler to a pure CGI handler with no > WebApp dependencies. The only imports are os, sys, datetime and hashlib > > During the high latency periods, this simple request takes 15-20 seconds > to execute. How can this be explained? So there are separate entries for each of these in app.yaml, correct? > > Now I have 4 different requests handlers that use > 1. Django 2. WebApp w/Django, 3. WebApp without Django, 4. CGI > and can clearly see how Django contributes to slow execution times. Here > are the average executions times (ms - pending_ms) for such types of > requests during the recent high latency period. > > Django : 132-300 seconds > WebApp w/Django : 22-50 seconds > WebApp without Django: 10 seconds (cron task, one dataset put()) > Pure CGI : 17-18 seconds > > Normally all requests take less than 1 second (less than 0.1 second for > WebApp and CGI) and their loading requests take less than 4 seconds even for > Django. > > So, Django is a significant overhead during the high latency periods. But > even simple pure CGI requests load unreasonably > sloooow (15-20 seconds for requests that only set a few headers and write > about 20 bytes of data). > > I don't like the idea of not using Django. The project will be hard to > maintain. At some point I may consider moving to a different framework, say > WebApp2 + Jinja2 or mako (by the way, any recommendations?), but this won't > solve the whole high latency issue. Now I think about migrating to HRD. Not > sure it will help though. I would suggest WebApp2 or flask + either Jinja2 or mako. I've been using webapp + mako for several years on App Engine; I have no complaints at all. I personally prefer mako syntax to jinja2's more django-esque syntax. I would suggest you setup a test app on HRD. It will almost certainly eliminate the issues you're seeing. > > P.S. Anyway, moving 60% of requests out of Django, helped reduce the > number of instances that App Engine loaded during the high latency periods. > Now the billing history pages show less instance/hours than I had before > those high latency periods :) Well, at least something good is coming from this issue. ;) > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/hoo6Iao5IxQJ. > > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
