2017-01-07 22:56 GMT+01:00 Daniel Dekany <ddek...@freemail.hu>:
> I have changed the subject so that we don't pollute the social media
> thread... See my comments below.
>
>
> Saturday, January 7, 2017, 9:01:06 PM, Christoph Rüger wrote:
> [snip]
> >> For example, apparently, you have hard time
> >> dealing with null-s in your application (lot of `exp!`-s everywhere),
> >> so you may want to chime in when the null/missing handling
> >> improvements in FM3 are discussed.
> >>
> > Yeah we just decided to put ! by default everywhere. In our uses cases in
> > 99% this is ok, as it should just render an empty String when something
> is
> > null.
>
> The reason we are picky about null-s is to catch typos. It's a partial
> solution as if you have values which can be legally null, you have to
> add some `!`, and then any typo you make will be hidden... but we
> couldn't do better if we assume that the data-model is some kind of
> Map.
>
Yes, I know. I think the strict null handling is ok. I also like it to spot
typos.
To make it short: I guess it is a bit of legacy on our end that we've
suggested the customer to put it.
>
> If you put `!` almost everywhere, then the way null is treated by FM
> is just a burden without the benefits. Can't you tell on that platform
> what the accepted variable *names* are? Because then you could default
> the valid ones to general-purpose-nothing (the result of null!) on the
> data-model level, and let the others be null and explode, because they
> are typos.
>
> BTW if I write ${nosuchthing}, it just prints nothing... So I do not
> need the `!` apparently.
>
I guess we are already experimenting with a wrapper underneath. I don't
know all the code anymore ;) But remember there are cases (maybe in other
parts... Transformy is just one part. we also use FM to write
filter-conditions for something we call SpreadsheetFilter...which you
cannot see in Transformy. maybe it was needed there because things can
become null... don't know).
I guess that is our part of the homework to figure out better
pre-processing here.
>
> >> BTW I see some examples like ${datecalc("HOUR", 1)!?datetime}. I guess
> >> you can't possibly get a null there. Even if you do, ?datetime will be
> >> unhappy with the '' it had to convert to date-time, unless you have a
> >> custom data-time format there.
> >>
> > Your are probably right. in this case the ! is not needed and we could
> > avoid it. Can't remember why we've put it... maybe just to make it look
> > consistent to our customers and to avoid questions ;)
> >
> >
> >>
> >> > (it"s german...just use google translator) where we show how to do
> >> > various stuff with Freemarker expressions. We have also written some
> >> > own functions (here) which could also be useful maybe as core
> >> > built-ins (see here)
> >>
> >> Some questions/notes...
> >>
> >> How does `escapeHTML(exp)` differ from `exp?html`?
> >>
> > our escapeHTML uses
> > https://commons.apache.org/proper/commons-lang/javadocs/
> api-2.6/org/apache/commons/lang/StringEscapeUtils.html#
> escapeHtml(java.lang.String)
> > under
> > the hood.
>
> Maybe because StringEscapeUtils.escapeHtml(String) escapes accented
> letters. Though nowadays that's not needed... you just set the
> response charset to UTF-8, and all funny characters come through.
>
Ok good to know.
>
> >> What's the practical use of `unescapeHTML(exp)`?
> >>
> > the opposite of escapeHTML using
> > https://commons.apache.org/proper/commons-lang/javadocs/
> api-2.6/org/apache/commons/lang/StringEscapeUtils.html#
> unescapeHtml(java.lang.String)
> >
> > I can't remember but I think we had a case where we needed exactly the
> > escaping of Apache StringUtils and maybe it was different than what
> > Freemarker does? not sure...
>
> FreeMarker doesn't have unescape functionality at all. The question
> is, why do you need to unescape in a template?
>
Not sure anymore, but I guess assume the following:
A 3rd party supplier of our customer sends a CSV containing escaped
HTML-code (with all the < etc.). But the customer wants to convert it
back to > because that is how he needs it in his output data (e.g. for
importing somewhere else). Most of the time our customers need to deal with
data which is a mess (like you said with Excel), but we cannot change it.
And we provide FM as the tool to move this data around until it fits your
needs. It does the job so far.
>
> >> Also note that there's the auto-escaping machinery (output formats,
> >> ?esc, etc.) nowadays. It tries to removing the responsibility of
> >> escaping from the template author (without using #escape, which itself
> >> can be forgotten, or had to be added with TemplateLoader hacks). You
> >> can have HTML fragments in the data-model that are automatically *not*
> >> escaped, and everything else is automatically escaped. But even if
> >> your data-model can't be made that smart (i.e. HTML is also Space), at
> >> least you escape be default (and the you can do exp?noEsc to prevent
> >> it), which is safer than the opposite.
> >>
> > Yes, we want to try that out too, but many of our custom functions were
> > created a long time ago. We currently have a strong focus on ecommerce
> and
> > onlinemarketing, so the use-cases which drove those functions where
> coming
> > from such projects.
>
> As far as those are #function-s or TemplateMethodModel-s, I don't
> think that it interferes with them. The values of those has to be
> escaped, I suppose. (Even if there are legacy macros, you can
> configure FreeMarker to not use auto-escaping for the templates the
> define the macros, and they remain callable from auto-escaping
> templates without problem.)
>
Ok, we will see once we start playing around with it.
> >> As of `now()`, we have `.now`, though that returns a java.util.Date
> >> instead of a long. So the equivalent is `.now?long`, which is somewhat
> >> ugly. Note sure why and how often do you need the numerical version
> >> though.
> >>
> > We provide both to the users in some kind of context-menu. Maybe at the
> > time we wanted something as short but with the numerical version.
> >
> >
> >> > Just let me know if this is interesting, so we can think about
> >> > concrete stuff we could contribute where it fits.
> >>
> >> Calendar arithmetic is something users often request (i.e., adding a
> >> day, truncating the date down to the beginning of the month/day/hour
> >> and such). Anyway, if you have some concrete recommendation for
> >> built-ins, tell us.
> >>
> > Yes calendar arithmetic is a big one. we already have a dateCalc()
> function
> > and first/lastDayOfMonth() (which we will probably make more generic into
> > something like firstDayOf("year",mydate) or so.
>
> Or something like PostgreSQL date_trunc(text, timestamp).
>
ok.
I like the chainability of Joda-Time API e.g.
*${mydate?plusDay(1)?string("yyyy-MM-dd")}*
*${mydate?date_trunc("month")}*
> > Customers need this often to create date-ranges e.g. to make API-calls
> to a
> > REST-API (e.g. "give me all orders from last month")
> >
> >> I'm interested to hear what your major pain points with FM are. A
> >> single template engine can't be ideal for everyone, but I'm hoping to
> >> cover more kind of usages in FM 3. (Or even in FM2, though I think
> >> things will be worked out in FM 3 and then maybe ported back where
> >> possible and if we have the resources)
> >
> >
> > Pain points...hmm.. maybe I would rather call it wishes, as I am
> generally
> > very happy with it. We really like the flexibility as it allows us to do
> > almost everything we want and also extend it. We even use it to parse XML
> > and JSON. The TemplateMethodModels are great.
> >
> > *my wishes would be (for the moment):*
> > - be able to create custom built-ins (like TemplateMethodModels but with
> > the ?mybuiltin syntax). This would allow us to build some custom
> functions
> > with the same syntax and it could be used like a fluent-interface
> > ?funcA?funcB?funcC
>
> That's a frequently required thing, but it raises some tricky
> problems. The whole point of built-ins was that since the `?` syntax
> is reserved for the template language, we can add new built-ins
> without breaking backward compatibility. And another advantage of them
> is that you can look them up in the FreeMarker Manual, because nobody
> can add/remove them.
>
> For FM 3, I'm thinking about introducing #import namespace prefixes
> that use `:` instead of `.`. So then, you would write `<#my:foo />`
> instead of `<@my.foo />`. Yes, that doesn't help so far, but it opens
> the possibility of writing x?my:bar, while with `.` it couldn't work
> (`x?my.bar`, which just means `(x?my).bar`). So this allows you to use
> the `arg?f` syntax instead of `f(arg)`), but only if you are using a
> namespace prefix (`my`), so that it's clearly separated from the core
> callables.
>
> However, some, especially in projects like yours, doesn't want to
> burden the user with such a prefix (`my:`) either. You just want to
> write `x?bar`. So that's also a valid use case, if you provide a
> complete documentation for your users instead them trying to look up
> things in the FreeMarker Manual. Because the problem with is not
> technical; it's the seer chaos this can start. Imagine if every second
> project starts to define his own FTL dialect, because now they can do
> it easily, without forking... My thought on this is that you should be
> allowed to define a custom FTL dialect, but then you aren't allowed to
> call it FTL or a FreeMarker Template. It would be Synesty Template
> Language (*.stl) or whatever. So then, you have your own specialized
> language, no silly prefixes, just what your users need (also no core
> FreeMarker functionality that you don't want to expose to your users),
> yet you didn't have to fork, and you benefit from the new FreeMarker
> releases. So I want the FM 3 architecture to be able to handle this
> task, but there are some non-technical issues with it; we might open
> some a Pandora's box be allowing this.
>
> This sounds pretty interesting and opens up new possibilities. I can see
this might be useful for us.
If we could define our own namespace globally also one idea could be to
define some kind of default namespace like you can do with FM's
XML-namespace
<http://freemarker.org/docs/xgui_imperative_learn.html#autoid_149>
handling:
<#ftl ns_prefixes={"D":"http://example.com/ebook"}>
What you say about the chaos makes total sense, but I still like the idea
of adding stuff on top of the FM-core-functionality.
Maybe it could be configurable somehow e.g. in pseudo-code :
*templateDialectMode=fm-core* (as it is today...only FM-core, nothing else)
*templateDialectMode=extend* (FM-core + own functions using a prefix.)
*templateDialectMode=onlyOwnNamespaces *(no FM-core, only own namespaces)
The latter would be great if you want to provide an (sandboxed) environment
with kind of a custom DSL (domain specific language) which should be very
limited.
I think the general rule should be: FM-core functions are not customizable,
but you should be able to add on top of it.
> > - date-arithmetic (see above)
>
> Yip...
>
> > - it would be cool if *?number* would handle trailing/leading spaces
> > instead of throwing exception (maybe as a configurable option) e.g.
> *Can't
> > convert this string to number: "1 " *
>
> Well... why not, I wonder, though I think it's might not be the right
> place to do that. Those corrupt strings are coming from somewhere.
>
Yes they are coming from all crazy places :)
>
> > - some magic to somehow being able to avoid ?number at all when you know,
> > that your strings are numbers. not sure how possible this is but the
> > use-case is the following: the data is usually coming from a CSV-file
> which
> > is parsed and the columns are available as freemarker variables (as
> > strings). It would be great if one could just do ${quantity * price}
> > instead of ${quantity?number * price?number} But of course, maybe we
> could
> > do that maybe also under the hood already when populating the
> > datamodel...just thinking...
>
> Oh, CSV-s... because users tend to drop them together with Excel, they
> are always full of random mistakes. I think that, generally, if you
> don't validate them when the user uploads them, you are in trouble.
It's not fair to expect the last step in the chain to manage that...
> (-: But at the very least, can't you trim anything you load from CSV?
> Why would an user want to enter leading or a trailing spaces?
A valid point :) Not sure: maybe it is our general thinking to not touch
the data the customers are pulling, because we don't know what a customer
is doing with it. But it could be actually a config-option for the
CSV-parser step.
> And I'm
> not sure if it can be practical in your application, but can't you
> require the columns of the CSV to be declared somewhere, together with
> the type of the columns? So you can expose numbers as a numbers, dates
> as dates (because what about them... try to auto-detect that), etc.
>
Yeah we have such a feature called Schemas where you could define data
types which is for more advanced use cases.
I get your point, I think that is something we need to do on our end. The
thing is just, that FM does the job in 99% of the time well, so sometimes
you come to ideas like ..."well it would be cool, if ?number could just be
a tiny bit more tolerant.." ;)
We are always trying to make it as easy as possible for the user, so that
things work out-of-the box without the need to define something upfront. So
it is sometimes a not so easy question which step in the chain should do it
:)
But no need to go into more detail here. I think this is on our end.
> Also, what's with `+`? It's overloaded, as for numbers it does
> arithmetical addition, but for strings it does concatenation. So I
> can't even know that I should coerce the string value to a number.
>
ah right..
>
> > as I said just an idea. It would maybe make some use-cases easier, but of
> > course you would need to be careful to not add too much magic too FM. I
> > also like it that freemarker is kinda strict, because it makes it easier
> to
> > spot errors. But in our world we are also exposing it to non-technical
> > endusers and there we need some compromises.
>
> Actually, they need the errors to be automatically spotted more than
> programmers do.
Agreed.
>
>
> Just take it as ideas :)
> >
> > If you want I can collect more stuff and ask our team to collect some of
> > their wishes.
>
> Sure, I'm interested in that.
>
I guess your namespace/dialect thing is THE feature and could help a lot so
that we are able to do things the way we want and still benefiting from the
base and syntax FM provides.
I see FM and especially the syntax as a very cool - let's call it
framework. Once we have tought our customers how the general syntax works
(e.g. ${var1} foo ${var2} is easy to get and also ${mystring?lower_case} is
easy to understand and <#if> <#else> </#if> too) everybody can use it. I
also like the ability to chain built-ins ?b?c?d this is cool and easy to
understand even for non-techs. So I think it would be cool to build on that
ground. TemplateMethodModels are great, but their syntax looks different
which sometimes is ok, but sometimes the ?builtin syntax would be nicer.
for example *${dateCalc(mydate, "DAY", 1}* vs. *${mydate?s:plusDay(1)}*.
I'd prefer the latter (I already used the s: prefix for e.g. our own
dialect).
> > Do we have a place to put stuff like this?
>
> Not yet... just post it.
>
Ok.
>
> > Christoph
> >
> >>
> >> > I don't want this to be considered spam - so let me know what you
> >> > think... but of course it would be promotion for us too
> >> >
> >> > Christoph
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > 2017-01-07 13:36 GMT+01:00 Daniel Dekany <ddek...@freemail.hu>:
> >> > Saturday, January 7, 2017, 9:37:36 AM, Denis Bredelet wrote:
> >> >
> >> >> Hi Daniel,
> >> >>
> >> >>>> That was also my impression on the OFBiz tweet, as a software
> >> >>>> developer. The last is important, that I'm looking at it as a
> >> >>>> developer. FreeMarker, unless OFBiz, is mostly only interesting for
> >> >>>> them I believe. It doesn't have an UI that a manager type could
> click
> >> >>>> around. It's not a complete end-user product, it's Java library
> used
> >> >>>> internally by other products.
> >> >>
> >> >> I am thinking of contributing a very simple UI for FreeMarker.
> >> >> Something able to read a single template and process it with a
> >> >> datamodel pulled from various sources (user interface window,
> >> properties file, XML file…)
> >> >>
> >> >> Would that be useful and help with advertising FreeMarker?
> >> >
> >> > I believe it's something that many users need, because it's not always
> >> > convenient to try things right in the real product that integrates
> >> > FreeMarker. Especially if you just want the wrap your head around
> >> > something you have just read in the Manual. So I was considering
> >> > creating a such GUI, only my priorities (core maintenance) didn't
> >> > allow that. And then suddenly come
> >> > http://freemarker-online.kenshoo.com/, so we do have something like
> >> > that, but it could be improved further, also perhaps an offline
> >> > version (like a SWT application or such) can be handy (if you want to
> >> > connect to a local DB, etc).
> >> >
> >> > These template evaluators are also useful to draw the user's attention
> >> > to best practices and features. Like if you go to
> >> > http://freemarker-online.kenshoo.com/, you will be aware of output
> >> > formats, a relatively novel but very useful feature.
> >> >
> >> >> I have some code to use as a starting point on my website:
> >> >> http://bredelet.com/Denis/FreeMarker%20first%20steps.html
> >> >
> >> > These are some getting started examples. What are your ideas about the
> >> > final product?
> >> >
> >> >> Maybe this could come as a separate JAR to avoid loading UI classes
> >> unnecessarily.
> >> >
> >> >> Cheers,
> >> >> — Denis.
> >> >
> >> > --
> >> > Thanks,
> >> > Daniel Dekany
> >>
> >> --
> >> Thanks,
> >> Daniel Dekany
> >>
> >>
> >
> >
> > --
> > Christoph Rüger, Geschäftsführer
> > Synesty <https://synesty.com/> - Automatisierung, Schnittstellen,
> Datenfeeds
> > Tel.: +49 3641/559649
> >
> > Xing: https://www.xing.com/profile/Christoph_Rueger2
> > LinkedIn: http://www.linkedin.com/pub/christoph-rueger/a/685/198
> >
>
> --
> Thanks,
> Daniel Dekany
>
>
Ciao
Christoph
--
Christoph Rüger, Geschäftsführer
Synesty <https://synesty.com/> - Automatisierung, Schnittstellen, Datenfeeds
Tel.: +49 3641/559649
Xing: https://www.xing.com/profile/Christoph_Rueger2
LinkedIn: http://www.linkedin.com/pub/christoph-rueger/a/685/198
--
Synesty GmbH
Moritz-von-Rohr-Str. 1a
07745 Jena
Tel.: +49 3641 559649
Fax.: +49 3641 5596499
Internet: http://synesty.com
Geschäftsführer: Christoph Rüger
Unternehmenssitz: Jena
Handelsregister B beim Amtsgericht: Jena
Handelsregister-Nummer: HRB 508766
Ust-IdNr.: DE287564982
