Re: Thoughts on using FM with Transformy (Was: Using Social Media to Help Promote Freemarker)

[Christoph Rüger]

2017-01-08 15:00 GMT+01:00 Daniel Dekany :

> Sunday, January 8, 2017, 1:10:11 AM, Christoph Rüger wrote:
>
> [snip]
> > ok.
> > I like the chainability of Joda-Time API e.g.
> > ${mydate?plusDay(1)?string("-MM-dd")}
> > ${mydate?date_trunc("month")}
>
> Or, since there's ?plus_day(n) (instead of ?date_plus("day", n)), it
> should be ?date_trunc_month(1). Or the other way around (both should
> have a parameter like "day" and "month"), but typos in that string
> aren't spotted until runtime, so maybe the first approach is better.
>
> Also instead of ?plus_day, ?add_day is perhaps marginally better,
> because ?plus_day(-1) look a bit stranger than ?add_day(-1).
>

All good ideas. I got the plusXXX from the Joda-Time API guys e.g.
http://www.joda.org/joda-time/apidocs/org/joda/time/
DateTime.html#plusDays-int-
I am fine with ?add_day(1) too...and week,month, year :)



> [snip]
> >>> *my wishes would be (for the moment):*
> >>> - be able to create custom built-ins (like TemplateMethodModels but
> with
> >>> the ?mybuiltin syntax). This would allow us to build some custom
> functions
> >>> with the same syntax and it could be used like a fluent-interface
> >>> ?funcA?funcB?funcC
> >>
> >> That's a frequently required thing, but it raises some tricky
> >> problems. The whole point of built-ins was that since the `?` syntax
> >> is reserved for the template language, we can add new built-ins
> >> without breaking backward compatibility. And another advantage of them
> >> is that you can look them up in the FreeMarker Manual, because nobody
> >> can add/remove them.
> >>
> >> For FM 3, I'm thinking about introducing #import namespace prefixes
> >> that use `:` instead of `.`. So then, you would write `<#my:foo />`
> >> instead of `<@my.foo />`. Yes, that doesn't help so far, but it opens
> >> the possibility of writing x?my:bar, while with `.` it couldn't work
> >> (`x?my.bar`, which just means `(x?my).bar`). So this allows you to use
> >> the `arg?f` syntax instead of `f(arg)`), but only if you are using a
> >> namespace prefix (`my`), so that it's clearly separated from the core
> >> callables.
> >>
> >> However, some, especially in projects like yours, doesn't want to
> >> burden the user with such a prefix (`my:`) either. You just want to
> >> write `x?bar`. So that's also a valid use case, if you provide a
> >> complete documentation for your users instead them trying to look up
> >> things in the FreeMarker Manual. Because the problem with is not
> >> technical; it's the seer chaos this can start. Imagine if every second
> >> project starts to define his own FTL dialect, because now they can do
> >> it easily, without forking... My thought on this is that you should be
> >> allowed to define a custom FTL dialect, but then you aren't allowed to
> >> call it FTL or a FreeMarker Template. It would be Synesty Template
> >> Language (*.stl) or whatever. So then, you have your own specialized
> >> language, no silly prefixes, just what your users need (also no core
> >> FreeMarker functionality that you don't want to expose to your users),
> >> yet you didn't have to fork, and you benefit from the new FreeMarker
> >> releases. So I want the FM 3 architecture to be able to handle this
> >> task, but there are some non-technical issues with it; we might open
> >> some a Pandora's box be allowing this.
> >
> > This sounds pretty interesting and opens up new possibilities. I
> > can see this might be useful for us.
> > If we could define our own namespace globally also one idea could
> > be to define some kind of default namespace like you can do with FM's
> XML-namespace handling:
> > <#ftl ns_prefixes={"D":"http://example.com/ebook"}>
> >
> > What you say about the chaos makes total sense, but I still like
> > the idea of adding stuff on top of the FM-core-functionality.
>
> I also want to go for it, but it remains to be seen what others will
> think about it. I guess as far as we communicate clearly when using a
> a custom dialect is appropriate, and as far as we try to enforce
> technically that people don't use ftl file extension, and show clearly
> in error messages that you aren't using FTL but STL or whatever (in
> case where it can have importance), it will be a much better
> compromise than what we have in FM 2.
>
> > Maybe it could be configurable somehow e.g. in pseudo-code :
> >
> > templateDialectMode=fm-core (as it is today...only FM-core, nothing else)
> > templateDialectMode=extend (FM-core + own functions using a prefix.)
>
> If you are using a prefix, then it's an auto-import (which you can do
> already, though not with `:` and so you can call those functions with
> `?`).
>

I guess you are referring to http://freemarker.org/docs/
dgui_misc_namespace.html when you say "already". Yes we use auto-imports
internally for some helper-macros in our templates. But they are not
exposed for users. We could, but at the moment we use TemplateMethodModels
for any additional 

Re: Thoughts on using FM with Transformy (Was: Using Social Media to Help Promote Freemarker)

[Daniel Dekany]

Sunday, January 8, 2017, 1:10:11 AM, Christoph Rüger wrote:

[snip]
> ok.
> I like the chainability of Joda-Time API e.g.
> ${mydate?plusDay(1)?string("-MM-dd")}
> ${mydate?date_trunc("month")}

Or, since there's ?plus_day(n) (instead of ?date_plus("day", n)), it
should be ?date_trunc_month(1). Or the other way around (both should
have a parameter like "day" and "month"), but typos in that string
aren't spotted until runtime, so maybe the first approach is better.

Also instead of ?plus_day, ?add_day is perhaps marginally better,
because ?plus_day(-1) look a bit stranger than ?add_day(-1).

[snip]
>>> *my wishes would be (for the moment):*
>>> - be able to create custom built-ins (like TemplateMethodModels but with
>>> the ?mybuiltin syntax). This would allow us to build some custom functions
>>> with the same syntax and it could be used like a fluent-interface
>>> ?funcA?funcB?funcC
>>
>> That's a frequently required thing, but it raises some tricky
>> problems. The whole point of built-ins was that since the `?` syntax
>> is reserved for the template language, we can add new built-ins
>> without breaking backward compatibility. And another advantage of them
>> is that you can look them up in the FreeMarker Manual, because nobody
>> can add/remove them.
>>
>> For FM 3, I'm thinking about introducing #import namespace prefixes
>> that use `:` instead of `.`. So then, you would write `<#my:foo />`
>> instead of `<@my.foo />`. Yes, that doesn't help so far, but it opens
>> the possibility of writing x?my:bar, while with `.` it couldn't work
>> (`x?my.bar`, which just means `(x?my).bar`). So this allows you to use
>> the `arg?f` syntax instead of `f(arg)`), but only if you are using a
>> namespace prefix (`my`), so that it's clearly separated from the core
>> callables.
>>
>> However, some, especially in projects like yours, doesn't want to
>> burden the user with such a prefix (`my:`) either. You just want to
>> write `x?bar`. So that's also a valid use case, if you provide a
>> complete documentation for your users instead them trying to look up
>> things in the FreeMarker Manual. Because the problem with is not
>> technical; it's the seer chaos this can start. Imagine if every second
>> project starts to define his own FTL dialect, because now they can do
>> it easily, without forking... My thought on this is that you should be
>> allowed to define a custom FTL dialect, but then you aren't allowed to
>> call it FTL or a FreeMarker Template. It would be Synesty Template
>> Language (*.stl) or whatever. So then, you have your own specialized
>> language, no silly prefixes, just what your users need (also no core
>> FreeMarker functionality that you don't want to expose to your users),
>> yet you didn't have to fork, and you benefit from the new FreeMarker
>> releases. So I want the FM 3 architecture to be able to handle this
>> task, but there are some non-technical issues with it; we might open
>> some a Pandora's box be allowing this.
>
> This sounds pretty interesting and opens up new possibilities. I
> can see this might be useful for us.
> If we could define our own namespace globally also one idea could
> be to define some kind of default namespace like you can do with FM's 
> XML-namespace handling:
> <#ftl ns_prefixes={"D":"http://example.com/ebook"}>
>
> What you say about the chaos makes total sense, but I still like
> the idea of adding stuff on top of the FM-core-functionality.

I also want to go for it, but it remains to be seen what others will
think about it. I guess as far as we communicate clearly when using a
a custom dialect is appropriate, and as far as we try to enforce
technically that people don't use ftl file extension, and show clearly
in error messages that you aren't using FTL but STL or whatever (in
case where it can have importance), it will be a much better
compromise than what we have in FM 2.

> Maybe it could be configurable somehow e.g. in pseudo-code :
>
> templateDialectMode=fm-core (as it is today...only FM-core, nothing else)
> templateDialectMode=extend (FM-core + own functions using a prefix.)

If you are using a prefix, then it's an auto-import (which you can do
already, though not with `:` and so you can call those functions with
`?`).

> templateDialectMode=onlyOwnNamespaces (no FM-core, only own namespaces)
>
> The latter would be great if you want to provide an (sandboxed)
> environment with kind of a custom DSL (domain specific language) which should 
> be very limited.
>
> I think the general rule should be: FM-core functions are not
> customizable, but you should be able to add on top of it. 

That won't work, because then adding a new FM-core function will break
backward compatibility.

[snip]
> I guess your namespace/dialect thing is THE feature and could help
> a lot so that we are able to do things the way we want and still
> benefiting from the base and syntax FM provides. 
> I see FM and especially the syntax as a very cool - let's call it
> 

Re: Thoughts on using FM with Transformy (Was: Using Social Media to Help Promote Freemarker)

[Christoph Rüger]

2017-01-07 22:56 GMT+01:00 Daniel Dekany :

> I have changed the subject so that we don't pollute the social media
> thread... See my comments below.
>
>
> Saturday, January 7, 2017, 9:01:06 PM, Christoph Rüger wrote:
> [snip]
> >> For example, apparently, you have hard time
> >> dealing with null-s in your application (lot of `exp!`-s everywhere),
> >> so you may want to chime in when the null/missing handling
> >> improvements in FM3 are discussed.
> >>
> > Yeah we just decided to put ! by default everywhere. In our uses cases in
> > 99% this is ok, as it should just render an empty String when something
> is
> > null.
>
> The reason we are picky about null-s is to catch typos. It's a partial
> solution as if you have values which can be legally null, you have to
> add some `!`, and then any typo you make will be hidden... but we
> couldn't do better if we assume that the data-model is some kind of
> Map.
>

Yes, I know. I think the strict null handling is ok. I also like it to spot
typos.
To make it short: I guess it is a bit of legacy on our end that we've
suggested the customer to put it.


>
> If you put `!` almost everywhere, then the way null is treated by FM
> is just a burden without the benefits. Can't you tell on that platform
> what the accepted variable *names* are? Because then you could default
> the valid ones to general-purpose-nothing (the result of null!) on the
> data-model level, and let the others be null and explode, because they
> are typos.
>
> BTW if I write ${nosuchthing}, it just prints nothing... So I do not
> need the `!` apparently.
>

I guess we are already experimenting with a wrapper underneath. I don't
know all the code anymore ;) But remember there are cases (maybe in other
parts... Transformy is just one part. we also use FM to write
filter-conditions for something we call SpreadsheetFilter...which you
cannot see in Transformy. maybe it was needed there because things can
become null... don't know).

I guess that is our part of the homework to figure out better
pre-processing here.



>
> >> BTW I see some examples like ${datecalc("HOUR", 1)!?datetime}. I guess
> >> you can't possibly get a null there. Even if you do, ?datetime will be
> >> unhappy with the '' it had to convert to date-time, unless you have a
> >> custom data-time format there.
> >>
> > Your are probably right. in this case the ! is not needed and we could
> > avoid it. Can't remember why we've put it... maybe just to make it look
> > consistent to our customers and to avoid questions ;)
> >
> >
> >>
> >> > (it"s german...just use google translator) where we show how to do
> >> > various stuff with Freemarker expressions. We have also written some
> >> > own functions (here) which could also be useful maybe as core
> >> > built-ins (see here)
> >>
> >> Some questions/notes...
> >>
> >> How does `escapeHTML(exp)` differ from `exp?html`?
> >>
> > our escapeHTML uses
> > https://commons.apache.org/proper/commons-lang/javadocs/
> api-2.6/org/apache/commons/lang/StringEscapeUtils.html#
> escapeHtml(java.lang.String)
> > under
> > the hood.
>
> Maybe because StringEscapeUtils.escapeHtml(String) escapes accented
> letters. Though nowadays that's not needed... you just set the
> response charset to UTF-8, and all funny characters come through.
>

Ok good to know.

>
> >> What's the practical use of `unescapeHTML(exp)`?
> >>
> > the opposite of escapeHTML  using
> > https://commons.apache.org/proper/commons-lang/javadocs/
> api-2.6/org/apache/commons/lang/StringEscapeUtils.html#
> unescapeHtml(java.lang.String)
> >
> > I can't remember but I think we had a case where we needed exactly the
> > escaping of Apache StringUtils and maybe it was different than what
> > Freemarker does? not sure...
>
> FreeMarker doesn't have unescape functionality at all. The question
> is, why do you need to unescape in a template?
>

Not sure anymore, but I guess assume the following:
A 3rd party supplier of our customer sends a CSV containing escaped
HTML-code (with all the  etc.). But the customer wants to  convert it
back to > because that is how he needs it in his output data (e.g. for
importing somewhere else). Most of the time our customers need to deal with
data which is a mess (like you said with Excel), but we cannot change it.
And we provide FM as the tool to move this data around until it fits your
needs. It does the job so far.


>
> >> Also note that there's the auto-escaping machinery (output formats,
> >> ?esc, etc.) nowadays. It tries to removing the responsibility of
> >> escaping from the template author (without using #escape, which itself
> >> can be forgotten, or had to be added with TemplateLoader hacks). You
> >> can have HTML fragments in the data-model that are automatically *not*
> >> escaped, and everything else is automatically escaped. But even if
> >> your data-model can't be made that smart (i.e. HTML is also Space), at
> >> least you escape be default (and the you can do