Re: Template Loader Implementation that loads from a database (DataSource)
I have only had the chance to look it on my phone, so have not read the thread thoroughly. I will go through it on my desktop later today. --Roberto Sent from Yahoo Mail on Android On Tue, Feb 7, 2017 at 8:14 AM, Daniel Dekany wrote: In case you have missed it in the other thread, note that it seems that there won't be a new alternative TemplateLoader interface in FM2, so this will have to be solved with the existing TemplateLoader there. -- Thanks, Daniel Dekany Wednesday, January 25, 2017, 4:17:10 AM, Roberto Benitez wrote: >>Also, I hope that freemarker-datasource-loader can be simplified over >>time, so it doesn't employ this many interfaces/classes. > Currently, the only essential classes are: 1)JdbcMetaData --A > central place to store DB table/column names. 2) TemplateSourceDao > --to perform the JDBC/CRUD work 3) DataSourceTemplateLoader -- the > loader implementation. 4) (Maybe)?) JdbcTemplateSource -- template > data record including the template content itself. This should be > helpful to a client such that no additional work should be necessary > (in most general cases) to maintain the templates > (name/content)--the DAO and JdbcTemplateSource will do the work. > > I am experimenting with the other classes/interfaces. In > particular, TemplateName and TemplateSource (which extends > TemplateName). The idea is to describe template metadata that is > more abstract that is more abstract than File, such that it could > describe a File or some database record--with the relevant data such > as name, locale. BUT, as you noted, necessitates pre-fetching data. > > > > >>My main problem with FreeMarker's TemplateLoader interface is that it >>assumes that you execute existence check, last modification query, and >>oading the actual content as 3 separate steps. While that fits how >>you deal with plain files (and I guess that's what it was modelled >>after), it doesn't fit how you do things with a database. In a >>straightforward implementation we would have 3 SQL round trips per new >>template loaded, and 2 for each up-to-date checks later. In a smarter >>implementation, and as you did it too, you can reduce the number of >>SQL operations be prefetching data into the templateSource object >>during the existence check (i.e., in findTemplateSource). As >>closeTemplateSource will be called pretty soon anyway, working from a >>such "cache" is certainly fine (because closeTemplateSource will >>invalidate it). But I don't think that you should prefetch the >>template content there, because then that will be an unnecessarily >>prefetched for template cache up-to-date checks, which is the dominant >>TemplateLoader usage in many applications. So it's sill 2 SQL-s per >>new template loaded, and 1 per template cache up-to-date checks. >>Certainly not a problem in most apps, but it's still bugs me. In >>FreeMarker 3 I definitely want to fix this be radically changing the >>TemplateLoader interface, and it's also possible that I will backport >>that solution into FreeMarker 2 (if people here will agree), though >>there it will co-exists with the traditional TemplateLoader, probably >>under the name TemplateLoader2 or something, so it's kind of messy. > > Would the changes to the TemplateLoader be that drastic? Sometimes > it is feasible to deprecate old/non-conformant methods (perhaps > provide defaults that rely on the new methods), which can later be removed > completely. > > --Roberto > > > >
Re: Template Loader Implementation that loads from a database (DataSource)
In case you have missed it in the other thread, note that it seems that there won't be a new alternative TemplateLoader interface in FM2, so this will have to be solved with the existing TemplateLoader there. -- Thanks, Daniel Dekany Wednesday, January 25, 2017, 4:17:10 AM, Roberto Benitez wrote: >>Also, I hope that freemarker-datasource-loader can be simplified over >>time, so it doesn't employ this many interfaces/classes. > Currently, the only essential classes are: 1)JdbcMetaData --A > central place to store DB table/column names. 2) TemplateSourceDao > --to perform the JDBC/CRUD work 3) DataSourceTemplateLoader -- the > loader implementation. 4) (Maybe)?) JdbcTemplateSource -- template > data record including the template content itself. This should be > helpful to a client such that no additional work should be necessary > (in most general cases) to maintain the templates > (name/content)--the DAO and JdbcTemplateSource will do the work. > > I am experimenting with the other classes/interfaces. In > particular, TemplateName and TemplateSource (which extends > TemplateName). The idea is to describe template metadata that is > more abstract that is more abstract than File, such that it could > describe a File or some database record--with the relevant data such > as name, locale. BUT, as you noted, necessitates pre-fetching data. > > > > >>My main problem with FreeMarker's TemplateLoader interface is that it >>assumes that you execute existence check, last modification query, and >>oading the actual content as 3 separate steps. While that fits how >>you deal with plain files (and I guess that's what it was modelled >>after), it doesn't fit how you do things with a database. In a >>straightforward implementation we would have 3 SQL round trips per new >>template loaded, and 2 for each up-to-date checks later. In a smarter >>implementation, and as you did it too, you can reduce the number of >>SQL operations be prefetching data into the templateSource object >>during the existence check (i.e., in findTemplateSource). As >>closeTemplateSource will be called pretty soon anyway, working from a >>such "cache" is certainly fine (because closeTemplateSource will >>invalidate it). But I don't think that you should prefetch the >>template content there, because then that will be an unnecessarily >>prefetched for template cache up-to-date checks, which is the dominant >>TemplateLoader usage in many applications. So it's sill 2 SQL-s per >>new template loaded, and 1 per template cache up-to-date checks. >>Certainly not a problem in most apps, but it's still bugs me. In >>FreeMarker 3 I definitely want to fix this be radically changing the >>TemplateLoader interface, and it's also possible that I will backport >>that solution into FreeMarker 2 (if people here will agree), though >>there it will co-exists with the traditional TemplateLoader, probably >>under the name TemplateLoader2 or something, so it's kind of messy. > > Would the changes to the TemplateLoader be that drastic? Sometimes > it is feasible to deprecate old/non-conformant methods (perhaps > provide defaults that rely on the new methods), which can later be removed > completely. > > --Roberto > > > >
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Tuesday, February 7, 2017, 7:40:18 AM, David E Jones wrote:
> On Tue, 2017-01-24 at 23:47 +0100, Daniel Dekany wrote:
>>
>> Should the TemplateResolver-s (the work name for the new interface)
>> still get the template name already resolved to absolute and
>> normalized? So when cfg.getTemplate is called, then it invokes
>> cfg.templateNameFormat.normalizeAbsoluteName(String) to normalize the
>> path, and only then it calls the TemplateResolver. Also #import and
>> such currently calls templateNameFormat.toAbsoluteName(String,
>> String), and that would happen earlier than TemplateResolver is
>> involved.
>>
>> And how would it look... perhaps like this:
>>
>> public interface TemplateResolver {
>>
>> Template getTemplate(String templatePath, Local lookupLocale, Object
>> customLookupCondition)
>> throws IOException;
>>
>> }
>>
>> You may notice that it misses the `encoding` and `parsed` parameter of
>> cfg.getTemplate. That's because I belive that in FM3 we should not
>> allow specifying those as getTemplate parameters anymore, but that
>> will be another thread when we get there.
>
> IMO it should pass through the exact text from the include/etc in
> the template. It might be worth constraining to a valid URI syntax
> but other than that a TemplateResolver would be much more flexible if no
> normalization/etc is done.
In FM2 there's a class like this:
public abstract class TemplateNameFormat {
...
abstract String toAbsoluteName(String baseName, String targetName) throws
MalformedTemplateNameException;
abstract String normalizeAbsoluteName(String name) throws
MalformedTemplateNameException;
}
My idea was that if we allow users to provide a custom implementation,
then if the standard rules don't fit your custom TemplateResolver, you
just create a custom TemplateNameFormat too. But now that I think
about it more, pushing the TemplateNameFormat functionality on the
TemplateResolver implementation does indeed look like a better
approach, because then it can't happen that a configuration
accidentally contains a TemplateResolver with a TemplateNameFormat
that doesn't work well with it. But then the above two methods should
be part of the TemplateResolver interface (because, for example, when
you #import something, FM has to find out if the same FTL was already
imported in the current Environment, so it needs to normalize the
#import argument without loading anything). WDYT?
> Passing in the Locale rather than trying to call it multiple times
> with different locale extensions is a good idea too as different
> underlying content or file stores may have their own approach for
> this.
That's even a problem with the TemplateLoader interface proposed to
FM3. Like in the recent database template loader thread, the author
wanted to use a separate column to indicate the locale of the template
(and the same could happen with custom lookup conditions too), so you
don't want to encode that information into the template name in the
case of that particular TemplateLoader. It's something we may try to
address in a second iteration of the FM3 TemplateLoader interface.
> For example a JCR back end has quite a bit for meta data and
> content variations.
>
> In general more parameters and less built into the path/name is a good thing
> IMO.
BTW, I still want to see if in FM3 the standard TemplateResolver (the
one that's basically FM2's TemplateCache, relies on TemplateLoader and
TemplateLookupStrategy and TemplateNameFormat) could cover more use
cases without becoming overly intricate. Also, even if you decide to
use a custom TemplateResolver, perhaps you still want to call some of
the stock TemplateLoader-s from it (while you replace the caching
logic, etc., or load from non-TemplateLoader sources too in case of
more sophisticated storages). I suspect you have experience with these
things, so it would be good if you can look at the FM3 TemplateLoader
thread.
> -David
--
Thanks,
Daniel Dekany
