Hey everyone,
I've been working on a patch for Django that would allow you to optionally cache templates after they've been lexed and parsed (compiled) by the template engine. I've got things far enough along that I have a working implementation, so I thought I'd share here and see if anyone had any thoughts / comments. I've attached my diff to ticket #6262, so visit http://code.djangoproject.com/ticket/6262 to check it out. I'd like to add template caching as a feature to Django 1.2, and am willing to do whatever is necessary to make this happen. If you'd rather read prose, here's a bit of an explanation / justification for the changes: Why? At the moment, Django is reading templates from disk, lexing them, parsing them, and then rendering them with the current context _every time_ a template is rendered. If your blog.html template loops through an array of 15 blog posts and {% includes %} post.html for each of them, post.html is read from disk (or disk cache), lexed, parsed, and rendered 15 times. All but the render step is avoidable with a bit of caching. I've done some rather crude benchmarking and, by my measurements, a template takes about 1ms to lex and parse. That doesn't seem like much, but it adds up. I'm working on a project that renders ~400 templates for the index page (you need to do lots of extends and includes if you're making a reusable app that you want to let people skin) and with this patch enabled we're seeing template rendering time decrease by about 350ms. Big win. How? The only thing that's absolutely necessary to make cached templates possible in Django is to add a branch in ``django.template.loader.get_template()`` that checks whether the returned template "source" is actually a compiled template, in which case it is returned directly and the compilation step is skipped. Once this is done, template caching can be implemented with a custom template loader. By simply checking that the template has a ``render`` method in ``get_template``, we get the added benefit of allowing users to write loaders that return custom Template instances, or Template instances that use an alternative template language like Jinja or Mako. Template-Tag State: In order to make cached templates usable in practice (and backwards compatible) some changes need to be made in the template tags as well. In particular, the block, extends, and cycle template tags store state on instances. Since template tags are instantiated when the template is parsed, this state sticks around between template renders. This also means templates are not thread-safe (a separate, but related problem) -- if two threads share the same instantiated template and both try to render it problems can arise. To make template tags safe for cached templates (and thread-safe) all state should be stored in the template context. But if we just stick state in the context dictionary we're polluting the template namespace with irrelevant stuff that probably shouldn't be exposed there. Therefore, I propose adding an attribute to Context instances, ``parser_context`` that can be used to store parser state. Unlike the Context object, I believe parser context should be statically scoped per Template render. That is, if the dictionary at the top of the "stack" doesn't contain the requested key, a KeyError should be raised immediately rather than walking the stack looking for the key in dictionaries further up the stack. I think this makes sense since parser state is generally associated with a single Template, and Templates are rendered recursively (because of extends and include tags). My implementation pushes() the parser_context stack at the beginning of each template render, and pops it after rendering is complete. Unfortunately, since there are numerous template tags that exist "in the wild," some work will probably have to be done to port existing Django projects over to use cached templates. But since template caching will be implemented using a custom loader, and off by default, users can choose to enable it or not, so this shouldn't be a problem. Refactoring Loaders: At the moment, template loaders are implemented as module-level functions. This makes them difficult to extend. I suggest refactoring the built-in template loaders to be classes. By implementing __call__ in the ``BaseLoader`` and instantiating a module-level instance of each loader we can maintain backwards compatibility. Cached Template Loader Options: Once all this groundwork is done, we need to decide how to implement a caching template loader. There are several options: 1. Don't include one at all. Let users write their own and implement it however they want. 2. Implement a generic caching loader that can be instantiated with a list of loaders that it should try to use to load templates, caching the results. This requires a bit of a change to ``django.template.loader.find_template_source()`` since the current implementation assumes you're passing in a string containing the module path. 3. Implement a caching version of each existing template loader. I've implemented option 2, and I think it's probably the best alternative. Looking forward to hearing your comments, suggestions, ridicule, etc, - Mike P.S. Thanks to Alex Gaynor and Marty Alchin who have helped review some of the stuff I've done so far. Respect and whatnot. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~----------~----~----~----~------~----~------~--~---
