I have noticed that the app cache does not keep things around for much more than a few seconds, so if your site gets relatively little traffic it will have more problems than if it gets heavier traffic (paradoxically). Also, if your app returns pages with many images or other embedded content that it also served by GAE then when the browser does a multithreaded set of requests GAE then forks off more processes to handle these which in turn also reload your app. (At least this is what I've gathered looking at the logs and the number and timing of zipimport invocations). Basically, using a large framework with GAE like Django 1.0 (when it's imported at startup via zipimport) is sort of unusable right now if your site gets light traffic and does some non-trivial stuff on the server side.
I understand the hesitancy to bundle a particular framework with GAE, but the GAE api, the webapp framework, and the bundled Django .96, all seems to indicate that Django was a major influence and I suspect many if not most of the applications use Django. It would make sense to bundle an up to date version of Django just as a default framework, non-Django users can always use whatever framework they want. And I don't buy the argument that upgrading the Django .96 release will cause a lot of grief, it really is not that hard to upgrade an application to use a more recent version. On Feb 9, 12:32 pm, johnP <j...@thinkwave.com> wrote: > Another question: where does one start to troubleshoot if caching > seems to fail. In my case (using appengine patch) - the app.yaml > handler points to a script: common/appenginepatch/main.py This > module *does* define a main() script. > > I placed logging.debug calls at the top of 5 common modules. When I > click through 5 different views (each defined in the different > modules) 5 times in quick succession, I see one instance of the > logging.debug evaluating for each of these modules in the logs. So > this tells me that something is caching, at least for this brief > period of time. > > But the interesting thing is if I wait 30 seconds or a minute. Then I > repeat the process (clicking through the 5 views in quick succession, > 5 times - for a total of 25 clicks). I see that the logging.debug > evaluates once again for each module - while the remaining 4 clicks > don't reevaluate. So it's like the interpreter goes cold in 30 > seconds. > > If it is caching for 20 seconds, shouldn't it cache for a minute? How > long should the cache last for? Any suggestions where to look > further? > > On Feb 9, 11:53 am, Andy Freeman <ana...@earthlink.net> wrote: > > > > Lazy importing (putting the import > > > statements directly in the code paths, instead of on module start-up) > > can > > help reduce the cost of that first per-server request, though that's > > not > > always easy to do when using a framework. > > > Hold it. I thought that the first instance was special in that it got > > extra CPU quota. (I don't remember if the "first instance" that gets > > extra CPU is the first invocation of a given handler in a process or > > the first handler used by a new process.) > > > If the "first instance" does get extra CPU, lazy import is a bad > > idea. Instead, the first instance should use that extra CPU to do > > things, such as import, that subsequent instances will/might need. > > > On Feb 9, 11:03 am, Dan Sanderson <dansander...@google.com> wrote: > > > > I can answer of couple of these: > > > > On Mon, Feb 9, 2009 at 9:29 AM, Mike Wesner <m...@konsole.net> wrote: > > > > 1. How does the import cache work exactly? I have read docs but still > > > > am left with questions on how exactly it works. If we have a main.py > > > > with our main() method in it... are all the imports in that file > > > > cached? Is there a way to keep our custom django in memory? > > > > All imports are cached on a per app server basis. The first time the app > > > imports a module on an app server, the module is loaded from disk and > > > evaluated. Subsequent imports of that module on that app server do not > > > load > > > or evaluate any code, in the same sense as if you imported a module more > > > than once in a single run of a Python application. > > > > If a handler script (a Python code file associated with a URL mapping in > > > app.yaml) has a main() method, the entire script is loaded and evaluated > > > the > > > first time the URL is requested. On subsequent requests for the URL, the > > > main() method is called without re-loading or re-evaluating the script. > > > This is similar to "importing" the handler script, with the minor > > > difference that the main() method is not called directly on the first > > > request, so you still have to call it in the script's code (see docs for > > > examples) for the first request to succeed. > > > > If a handler script does not have a main() method, it is evaluated in full > > > for every request, and its global variables are not retained on the app > > > server between requests. > > > > > 2. Since so many people use custom django, is there any plan to > > > > provide other versions of django besides the .96 version? Would it > > > > be possibe to provide .97 and 1.0 under some other name (import > > > > django97 as django)? I have read that the provided django is always > > > > loaded in memory and thus would be faster. It would sure save us a > > > > lot of files to upload also. > > > > So far we're mostly considering upgrading the bundled Django only for the > > > next major API version. With the way the module load path works today, > > > it'd > > > be difficult to introduce new modules into the default environment as a > > > backwards compatible change. As I've gone on about before here in the > > > group, having major third-party packages bundled with the API is not a > > > good > > > way to go in the long run, and I'd rather see this need addressed by > > > making > > > third-party packages easier to add to apps. > > > > App caching ought to be sufficient for mitigating the time spent importing > > > modules. Have you profiled your imports to see if they're a major > > > contributor to performance issues? Lazy importing (putting the import > > > statements directly in the code paths, instead of on module start-up) can > > > help reduce the cost of that first per-server request, though that's not > > > always easy to do when using a framework. > > > > -- Dan > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
