At 05:07 PM 2/3/2006 -0800, Ben Bangert wrote:
>On Feb 3, 2006, at 3:49 PM, Phillip J. Eby wrote:
>>I don't understand what you mean by "outside of Myghty" in this
>>context. My suggestion was simply that would gain pluggability in
>>one main area: the implementation of compiling and serialization of
>>templates. (i.e., the syntax/semantics of the templates used)
>
>I meant decoupled from the request to render the template. Since
>rendering the template may or may not require reloading/re-caching
>it, it seems like a process that the framework shouldn't have to
>consider.
My thoughts on caching were in the context of assuming the "manager" was
part of the framework. Frameworks that implement skins/layers and
localizable resources have to do this, because template systems generally
don't have any concept of skins or layers. But it's seeming like we'll
have to punt on this to some extent for now.
>>I'm still not understanding. Since Myghty will be in control of
>>all the streams, it will have full ability to monitor changes etc.
>>The only thing that's different under the embedding scheme is that
>>1: it would support multiple formats for the actual templates, and
>>2: it would return WSGI apps that frontend access to an individual
>>template.
>
>Yes, that is useful functionality. Would the call to the WSGI app
>that renders the template also trigger something that ensures its re-
>compiled if it needs to be?
Sure. You can just do something like:
def resource_returned_by_manager(environ, start_response):
if needs_recompiling(actual_resource):
recompile(actual_resource)
return actual_resource(environ, start_response)
That is, the manager doesn't have to return the resource returned by the
compiler; it can wrap a trivial reloader middleware around it and return that.
Of course, the more I look at this, the more I realize that your idea of
just putting the entire "manager" interface in one WSGI call might make
good sense too. Because unless the "publisher" is going to hang on to the
returned app, you might as well just integrate this stuff at the manager level.
The problem for me is, I was hoping we would end up defining a
manager->compiler interface and a publisher->resource interface rather than
a publisher->manager one. Then again, maybe we should define both, and let
people plug things in where they can. It would provide a path for
gradually evolving to the place where everything's pluggable.
>It usually is, which is part of the reason it might be nice to have
>it decoupled so the most robust compiler/loader can be used, if thats
>possible. Such template agnosticism might come at a very hefty
>workload cost, as Mike mentioned about how much time it'd take to
>splice this functionality into separate parts.
I'm leaning more now towards defining a publisher->manager and
publisher->resource interface pair based on WSGI as making the most sense
for the present. A manager->compiler interface would be nice too, but
perhaps not strictly necessary for the original use cases that prompted
this discussion.
>>I'm seeing now that the key problem with my view of the situation
>>is that I've been assuming *frameworks* have control over where and
>>how templates are stored, and considering the template systems to
>>be just that - implementation of the template itself.
>
>Ah, yes. That explains a lot. In my experience with various template
>systems, the directory structure of the template root can also
>influence the rendering of the template. In Myghty this structure is
>used for automatic inheritance, Cheetah, Kid, Django templates, and
>others all include the ability to "extend" existing templates within
>a template. This means they all need to resolve the included
>template, and cache it. That reason is why all of them already have
>their own caching/compiling stuff integrated.
In my view, this is a bit suboptimal in the long haul, because there's all
this duplicated code for managing files, and probably little -- if any --
of it works with eggs, or templates in a database (e.g. ZODB), etc. This
is part of why my mental model didn't really consider much about template
systems having their own "manager" parts - the idea seems manifestly
unworkable if you need to do things like what Zope 3 supports.
Of course, my long-term ambition here would be that we get to a place where
you can do skinning and localization with templates and other resources,
all using a Python-wide standard for how you identify and deploy these
resources. But that's out of scope for the current interface discussions,
alas.
It might be nice, though, to put together a session at PyCon to do some
brainstorming on templates, embedding, and deployment to see if there are
currently any easily-reached opportunities.
>>So, on a given request, the publisher asks the manager for a
>>resource by saying what template it wants. The manager does
>>whatever it needs to do, possibly invoking the compiler if it needs
>>to. The publisher then makes a WSGI call on the resource to render
>>it.
>
>Where along these steps, are the variables passed into the template?
When the publisher makes the WSGI call, it would add a 'params' key and/or
a 'published_object' key, depending on whether the publisher is an object
publisher or does parameters or both. I can imagine that Zope, for
example, might do both, drawing on its request parsing to do parameters. A
TurboGears or other controller-type framework that uses templates for
postprocessing would put the variables in 'params'. These names are
hypothetical and subject to getting a firm spec, of course.
>Once the template is compiled/cached, in addition to calling it, I
>need to pass it variables that it will be using. Are all templates
>going to be expected to pull variables they should see from the environ?
Yeah, from the 'params' key or 'published_object' or both.
>>I was at first assuming that "framework" meant the publisher and
>>manager, and "engine" was the compiler and resource. Then to
>>explain better I split compiler and resource, but now I see that I
>>should've split (conceptually) the publisher and manager.
>
>Ok, so if I'm using this in my web application. Somehow, I'm going to
>need to check to see if the template has been compiled/cached, then I
>have to go and call the WSGI app? With all these compiled/cached
>templates that've been used, what's holding onto all those many, many
>WSGI apps that've been created? Some of my web applications have
>upwards of thousands of templates, is there going to be thousands of
>WSGI apps in memory waiting to be called? I'm guessing/hoping I
>missed something and this isn't the case.
Nah, the issue here was that I envisioned the division differently than
Ian; my concern about being able to cache was because I envisioned the
"manager" being on the framework side of the line; ergo, the framework
needed caching. If the interface is strictly between the publisher and the
manager, then you could potentially just do it all with a single WSGI call,
in which the manager pulls the template identifier from the environ.
But it seems a little ugly to me to cram another key into environ to tell
the manager what template to use, when you could just have a
'get(template_id)' call that gets you the WSGI app to use. It also makes
the objects' responsibilities a little more clearly defined, since a WSGI
app in the general case doesn't take a template_id parameter, if you see
what I mean.
>Since templates in a bunch of these template languages will
>frequently include other templates, all the 'resource' instances will
>need access to call back to the 'publisher' so that they can do what
To the manager actually, since the manager is responsible for id->template
lookups.
>they need to for the template to be rendered and included. Some
>template systems go farther still than basic including of another
>template and start extending other templates. Kid will have one
>template override parts of templates it "inherits" from. Myghty will
>have parent templates that will insert their "child" templates in
>various fashions. Cheetah and Django also do things like this.
ZPT has macros, peak.web has DOMlets and layouts, etc., etc. Lots of
variety to choose from. :)
>All these systems will need ways to tie back into the manager so that
>they're all loaded, compiled, cached when needed, and that individual
>parts of the whole get reloaded when necessary without reloading
>everything.
>
>If there's a way to separate out this very convoluted process (that
>isn't necessarily the same for each system), it'd be very compelling
>I think. Especially since solid code to handle many of the problems
>that can occur under load when doing all this reloading/caching/etc
>is not easy stuff.
I'm beginning to think that this actually ties in pretty closely with the
deployment issues, in the sense that you may need basically multiple
"sources" of template/resource implementations, especially if you're
supporting skins or localization. It's almost like you need a registry of
template ids to ways to get them, that have been registered via plugins.
Of course, in the use cases you guys are talking about, you presume that
there's just a "template root", which is way too simple for the kinds of
stuff Zope 3 and peak.web want to do. Both have the notion of "skins",
which are composed of "layers". A layer can provide alternate versions of
any given template, and layers can be combined to form skins. There's no
particular requirement that these layers have any filesystem relationships,
and in the Zope case they may be in ZODB in any case, not the
filesystem. In PEAK's case, the layers can live in different eggs, and I
suspect Zope will want to allow the same as they increase their support for
eggs.
Anyway, that's why I think that if we can come up with a "bottom up" way to
provide template sources, then we can have a manager that's just a thin
layer over these providers. Stuff that's supplied by eggs, for example,
will probably be precompiled and won't need to be refreshed. Stuff in ZODB
has its own caching. So, to a certain extent, I think the ultimate future
will be that our "manager" will just be something that figures out what
backend is supplying the stuff and hand off the handling to it. We'll have
some "file" managers, "egg" managers, "ZODB" managers, etc. plugged in via
entry points or some such.
Of course, this is all "future" stuff. I had hoped we were closer to being
able to do it, though, as I'd been meaning to build some of it for
peak.web, and didn't want to end up in the position of appearing to say,
"hey, everybody should do this stuff I did for peak.web". :) Instead, I'd
be saying, "peak.web now supports the new WSGI embedding and resource
deployment specs". :)
>I think many of us are actually talking about two different
>proposals. The proposal for standardized templating you're talking to
>sounds like WSGI for template languages. That is, it pushes common
>parts many template languages have, into a re-usable layer. I'm not
>sure such a thing is possible, its definitely quite ambitious, and I
>look forward to the more clearly defined spec you put out.
Well, I'm realizing now that the more ambitious bits are harder because
there's more that's actually been done out there in the template engines
than I thought. If there were less, more standardization would've been an
obvious move forward and more doable now.
>The proposal many of us were looking for (I think), was merely a more
>unified way of calling a template. That is, we want a standard
>'publisher' interface, so that we can call someengine.render('some/
>template.myt', myvars) and its rendered. The TurboGears template plug- in
>thing describes such a thing, but it could use some more work. The
>way I interpreted your proposal originally also set things up in such
>a way that there was a base system for configuring a template engine,
>then you just call the app and put your vars in environ.
>
>As the more ambitious one is likely to take significantly more effort
>from everyone involved (assuming it'll work), can we unify a setup/ call
>structure first? I see no reason it won't be compatible in the
>end with standardizing the "internals" of the template languages as
>well.
I concur. What I propose right now is that we have a "manager" interface
that gets asked for a template by a name or ID, and returns a WSGI app
object. Calls to that object can then include 'params' and/or
'published_object'. (Exact spelling TBD.)
I don't think this interface should suggest/recommend an addressing scheme,
nor a plugin scheme. That is, I don't think it should include specifiers
for entry points or anything like that. Instead, to do the TG/B style of
plugins, you should just have a manager that figures out what template
engine/plugin to use. The identifier can and should be opaque.
By that I mean, you could have a manager object that looks for a prefix
like "kid:" at the beginning of the ID and then looks up an entry point to
forward to another manager. But that would just be one way of implementing
the spec, not something required by the spec. The reason is that this then
leaves open both the possibility of developing generic and/or polymorphic
"manager" objects.
The big open question is how to create and configure a manager object,
since different managers will have different needs. For example, peak.web
uses module-relative paths in the default layer of the default skin; it has
no "root" directory. Each layer can also have its own directory, and I
believe Zope 3 does the same.
_______________________________________________
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe:
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
