Hi Malthe On Thu, Feb 4, 2010 at 8:53 PM, Malthe Borch <mbo...@gmail.com> wrote: > On 4 February 2010 13:48, Tim Hoffman <zutes...@gmail.com> wrote: >> Do you mean it parses all templates on instance startup. If so that >> would be prohibitively expensive >> if you have a lot of templates. > > Yes. > >> And startup time (just processing all >> the zcml, and all the other imports) is problematic as it is. >> (In the project just compeleted www.polytechnic.wa.edu.au, there is >> 131 template files. (page templates and metal macros). Our startup >> time > > You can use the CHAMELEON_CACHE=1 environment setting to cache these.
Are intermediate python files created in the filesystem normally. If so then they could be deployed and then the templates will always be in the filesystem on app engine as well. Which in fact would be fine and probably the best approach. Just need to then make sure that nothing tries to regenerate them on startup time as that will fail. (Which I assume couldn't happen) > >> for a cold instance without anything in memcache is between 4 and 8 >> secs. If appengine is having problems then that can blow out a very >> long way) > > It probably makes a lot of sense to memcache the templates, although > memcache might decide to expunge the cached files since they won't be > used much (only at startup). > >> Do you have any idea how much it costs to parse a template first time >> round vs traditional zpt. > > Chameleon takes substantially longer than ZPT to parse and compile templates. > So deploying the filesystem cache would be the way to go. Because even parsing/compiling them on demand is likely to be too slow. >> I am going to have to have a bit of play, but parsing all templates on >> startup has me really worried. Memcache would alleviate that somewhat >> but any new instance would then still have to check to see if every >> template exists in memcache on startup and parse missing ones. >> However the main problem with a memcache strategy on google is such >> that anything can disappear from it at any time, which means if it's >> gone from memcache then you would have to parse on use of the >> template. Which sort of counters the whole approach. I wonder if the >> datastore could >> be used as a first line storage for the compiled templates. The they >> could be loaded and then cached in memcache, that way the cost >> would not be too high if the are not present as the loader could grab >> them from the datastore. > > I thought just that, too. This approach might be good. Can you try it > out? Other than that, perhaps it's possible to fool memcached. > Sticking templates in to the datastore would be relatively easy. But if the filesystem cache is deployed that would be the best. That way you are sure to deploy current templates from svn at deployment time. Building/updating compiled templates in the datastore could be very prone to problems if you can't update it in a realistic time from and appengine also has versions so you would need to key all of the templates based on the versions etc..... All starts getting a bit messy. And unfortunately you can't fool googles memcache. About all you can hope is not exhaust the 250MB limit and keep everything active. But from what I have experienced with memcache behaviour on appengine you can't rely on it. (ie you really need it and it does a great job in helping you keeps things fast but you absolutely cannot rely on stuff to stay in it.) I will be playing with chameleon over the next few days and should have some definitive answers by mid next week. See ya T > \malthe > _______________________________________________ Repoze-dev mailing list Repoze-dev@lists.repoze.org http://lists.repoze.org/listinfo/repoze-dev
