On Fri, Nov 10, 2023 at 8:02 AM Jacques Le Roux < jacques.le.r...@les7arts.com> wrote:
> Hi Jonathan, > Salut, Jacques. > > What about existing projects using the Apache version ? > Well, what about them? The basic problem is here not very different from the decision to upgrade to a newer, not 100% backward compatible version of whatever, like moving from Python 2.x to Python 3.x. Now, one thing to be clear about is that in its out-of-the-box configuration, practical NO existing Apache FreeMarker template will work with the newer version. That's because, by default, directives like <#assign..> and <#local...> don't work. But if you put <#ftl legacy_syntax> up top in your template, then they do work. But anyway, the thing is that there are different levels of user. If you have a very basic usage pattern, like you just build up some data model that is a tree of hashes and maps ending in scalars (strings and numbers) and you expose that data model to your templates... well, the truth of the matter would surely be that there is very little difference between FreeMarker 2 and FreeMarker 3. Now, there are massive differences under the hood and this is now very largely a rewrite, but what I mean is that a user with a very simplistic usage pattern, (which actually could well be the majority of users!) just would likely not notice much difference. Though, again, they would have to use the legacy_syntax configuration or just about nothing will work! Since my announcement, I put up a new page which tries to gather together all the new features in one spot. That is here: https://github.com/freemarker/freemarker3/wiki/Summary-of-New-Features > > I mean the move from 2.3 version to 2.4. Like: > > 1. Would it be an easy move? > Well, again, this is a very nuanced question because there are different kinds of users. As I say, if you have a very basic vanilla usage pattern it probably is an easy move. In fact, most likely your templates will continue to work with no changes (or close to none) in the legacy_syntax mode, but even getting them to work without that is probably not so hard. But, of course, the flip side to that is that, yes, it would probably be quite easy to upgrade, but if your usage of the thing is that simple, then there may not be much benefit either! That's true enough. Now, on the other hand, if you are what is popularly called a "power user", really pushing the limits in terms of what the tool can do, then I would say that almost certainly you should try to move to the newer version. (Even if it will be harder initially.) That is for a variety of reasons. If you're a power user, and you are hitting limits in what the tool can do, and you suggest a new feature, well, let's face it. The likelihood of that new feature being implemented in "Apache FreeMarker" is extremely low. The version that is now being actively developed is FreeMarker 3, and if somebody has an idea that seems well motivated it is very likely that I'll implement it. But the other aspect is that the codebase is so much cleaner that it is easy now. I don't know what your level of familiarity with the code is, but you would likely know that the grammar/parser part was written using this rather old tool called JavaCC. FreeMarker 3 is written using CongoCC, which is a vastly more advanced version of JavaCC. CongoCC started as a fork of JavaCC but is now a total rewrite. But, to give you an example of what I'm talking about, that page I linked mentions various new features. Let's take the ternary operator as an example. Here is where it is implemented: https://github.com/freemarker/freemarker3/blob/master/src/parser/Expressions.inc.ccc#L95 or here, for example, is where the #assert and #exec directives are implemented: https://github.com/freemarker/freemarker3/blob/master/src/parser/Directives.inc.ccc#L417-L459 So, the point is that, what with using a much more powerful parser generator tool, and the codebase being so cleaned up, it is extremely easy to implement new features, certainly compared to trying to do it against the 2.3 codebase. So, in particular, for anybody who really aspires to develop more of a relationship with the code, the FreeMarker 3 codebase is really where it's at. That's clear enough. Again, I don't know what your understanding level of the code is, but there were things that were really a bear to deal with in the legacy codebase, like all this wrapping/unwrapping of variables. That's all gone. It also means that a lot of things do actually just work more simply because you're just working with Java objects, not all this wrapped TemplateXXXModel nonsense. So, when write: #var myList = [1, 2, 3] the object constructed is not some weird wrapper object. It's just a plain old java.util.List (an ArrayList to be precise) you can just write: #exec myList::add(1, "foo") #-- Look, ma! No brackets! And then your list contains [1, "foo", 2, 3] (Not that I'm even saying that the above is necessarily such a great usage pattern, but my point is that things are simpler because you're just dealing with POJOs finally. If you want to do that, you can because this is just a plain old Java object!) And the same thing applies to maps. The old bugaboo that the map's keys have to be strings and the various workarounds. That's gone. The maps are just Map<Object, Object> not Map<String,TemplateModel> (WTF is a TemplateModel anyway??!! LOL). So this refactored cleaned up version is much easier to work with really. But, you know, that said, all this overly complicated wrapping/unwrapping was implemented in a (kind of intricate) way that was mostly transparent to most users, so many people might not notice that this whole mess is cleaned up. Because they're unaware of the whole mess! It's a funny situation actually. But if you're a "power user" you'll surely notice that the whole thing is much cleaner now! > 2. What does it brings? > Well, like I said, you can consult the page that I put up recently. There are significant improvements. I think the terse syntax (even though that is actually technically superficial really) is quite an improvement and most anybody who has to spend much time editing templates will appreciate that. (All the more so once I can convince some tool makers to start providing some syntax highlighting at least! That, by the way, could be an easy way into the project because it's quite low-hanging fruit, I think.) > 3. What are the pros and cons of each version? > Well, the 2.3 codebase that is what "Apache FreeMarker" is, that's more "stable", sure. But that's a temporary situation. One thing is that the one activity that took place over the last so many years is that a lot of built-ins were added that never were in the 2.4 codebase. There are dozens of built-ins that are present in Apache FreeMarker that I haven't implemented. (Yet.) The approach I'm going to take, I think, is that I'm not going to make much of an active effort to implement every last missing built-in, but if people show up and say that they really miss a given one, and it's not terribly hard to add, I'll add it. Perhaps, in short order, FreeMarker 3 will have some built-ins that FreeMarker 2.x does not have. And the other thing is that, since the objects on the template layer are now pretty much all POJOs (plain old java objects) it is very very much easier to implement builtins than it ever was before, since you no longer need all this abstruse wrapping/unwrapping, so.... > 4. etc. > > I guess that's not easy questions to answer to (4 being somehow a joke ;), > but they are fundamental. > Well, they aren't such fundamental questions really, Jacques. The more advanced version is simply more advanced. Python 3 is simply more advanced than Python 2. JDK 8 is simply more advanced than JDK 7. And people will be better off using the more advanced version. You can get into a lot of sophistry trying to make the case that the less advanced version is somehow better and all that, but really, I wouldn't even care to engage much in such a conversation. So, to answer your question, it is dubious that a mature project that uses FreeMarker and has a big investment in macro libraries and so on, would opt to update at this point in time. But I would say that a new project really would be doing themselves a favor using the newer version of FreeMarker. It's just comparatively much more of a pleasure to use. I mean, for example, just take a look at the newer terse syntax. https://github.com/freemarker/freemarker3/wiki/Terse-Syntax Maybe it's not an absolute must-have, but it's clearly better, isn't it? And, in any case, it's optional anyway. The older syntax(es) still work. And the newer #set/#var is simply better than #assign/#local. In fact, that is on the FreeMarker 3 wish list that this community (Daniel, obviously...) put up. So nobody is contesting that #set/#var is better. Maybe you don't know about it, but read here and make your own judgment: https://github.com/freemarker/freemarker3/wiki/Strict-Vars But the other thing is that this is a moving target. The FreeMarker 2.3 codebase (which is what "Apache FreeMarker" is) is basically stagnant. If you commit to that in preference to the actively developed version, you are basically cutting yourself off from all the improvements that are going be coming along. The code has been cleaned up to such an extent that things that were very hard to do with the older FreeMarker code are very easy to implement now. As an actively developed project to get involved in, well, obviously FreeMarker 3 is where it is at. Anyway, that's enough said for now. Look, Jacques, if you have more questions, probably they're not enthusiastic about this sort of conversation here, so maybe it would be better to start a discussion here https://github.com/freemarker/freemarker3/discussions or here is good too: https://discuss.congocc.org/ A bientôt, j'espère Jonathan Revusky > > TIA > > Jacques > > Le 10/11/2023 à 02:50, Jonathan Revusky a écrit : > > On Thu, Nov 9, 2023 at 9:00 PM Benjamin Marwell<bmarw...@apache.org> > wrote: > > > >> I never knew there was an "original" freemarker project. > >> > > So you actually thought that FreeMarker was developed at Apache? > > > > Well, no. FreeMarker is a very very old project at this point. > FreeMarker 1 > > was originally written by a guy named Benjamin Geer, in the late 90's. > > Though Ben Geer was, strictly speaking, the original author, I don't > think > > he was really involved in the project for very long. He wasn't involved > > when I showed up in the community in late 2001 anyway. At that point, I > > basically took over, and within a few months, the thing was a complete > > rewrite. And that was when FreeMarker 2.0 came into being. From 2002 to > > 2004/2005 we went through 4 release cycles, 2.0, 2.1, 2.2, and 2.3. Each > > release cycle added quite a bit more functionality. It is kinda sad that > > there is just about no meaningful difference between 2023 "Apache > > FreeMarker" and FreeMarker 2.3 from 2005 (or even late 2004). > > > > But the thing to understand is that this Apache FreeMarker code, the > > continuation of the FreeMarker 2.3 codebase, is really something very > > ancient. Most of the work on this was done in the period from 2002 to > 2005 > > or so, about a decade before there was any "Apache FreeMarker". > Basically, > > the project was very old and stagnant at that point and came to Apache to > > die, I guess. So I've decided to resuscitate it. Or give it my best > shot... > > > > > >> Your web site is down, the documentation on the GitHub project is > sparse. > >> > > That is true at the moment but is all quite remediable -- especially if > > some people want to get involved and do some heavy lifting. (I get the > > feeling that's not you!) In any case, I said quite clearly that this is a > > preview. You can't expect something that is a preview to be as polished > as > > something as old as FreeMarker 2.3, which has been pretty stable since > 2004 > > or thereabouts! > > > > > >> There is no way to tell whether it really is more advanced or not. > >> > > Well, frankly, this is just nonsense. There is no legitimate controversy > > over whether this version of FreeMarker is more advanced or not. Of > course > > it is. As I explained in the previous note in response to Taher > Alkhateeb, > > it is built on top of the 2.4 codebase, while Apache FreeMarker is a > > continuation of the 2.3 codebase. Aside from that, just scan over the > > commit record:https://github.com/freemarker/freemarker3/commits/master > > Truth told, over the last few months, I have effected something close to > a > > complete rewrite. > > > > But this is just ridiculous! Tell me, do you think there is some > legitimate > > controversy over whether JDK 8 is more advanced than JDK 7? That's just > > silly. In any case, both FreeMarker 2.3 and this FreeMarker 3.0 preview > > that I just announced are largely my work. Is it possible that an earlier > > version of work by the same author is more advanced than the later > version? > > Does that make any sense? Of course this version is more advanced! > > > > It can never be on Maven central, because the namespace (groupid) > >> "freemarker" is already claimed by Apache Freemarker. > >> > > Well, Ben... it is kind of disrespectful to talk such blatant nonsense to > > somebody. This is supposed to be some serious technical forum, isn't it? > > > > The "groupid" used on Maven Central is not something with any real > > transcendence at all. It certainly has no technical meaning. I mean, > look, > > here is an example. The main OSS project I'm working on, as I said > before, > > is CongoCC. A few months ago, our project (finally!) put out an > "artifact" > > on Maven Central. That is here: > > > https://central.sonatype.com/artifact/org.congocc/org.congocc.parser.generator > > I later realized that somebody else had previously put up a Maven > artifact. > > That is here: > > https://central.sonatype.com/artifact/com.clickhouse/org.congocc Funny > > enough, the guy who put that up was not even in touch with us about it > > beforehand. But the one we put up is, I guess, under org.congocc and the > > one put up earlier by a third party is under com.clickhouse, which I > guess > > is the URL he controls or his employer, or... I dunno... Actually, I just > > looked, and there is a patched version of FreeMarker 2.3.29 put up by > > Liferay, which is this one: > > https://central.sonatype.com/artifact/com.liferay/org.freemarker > > > > But the point is that it just doesn't matter! The whole idea that I can't > > put something on Maven Central because this nothingburger project > controls > > the freemarker.org domain... Well, okay, I guess it's true that we can't > > use "org.freemarker" as a groupid since it's taken but... so what? So we > > use something else. (Duh.) When I decided on CongoCC as a new name for > the > > parser generator project, I checked whether congocc.org was available > and > > registered it. But I had anticipated having github.com/congocc as our > > "organization" location, but somehow that was taken, so we use > > github.com/congo-cc with a hyphen. Whatever. It would be a bit nicer if > we > > had congocc without the hyphen, but it's hardly a sine qua non either. > > > > Well, anyway, look, we're not acquainted, but I find this quite > > off-putting. You have the possibility of raising whatever technical > issue, > > making suggestions, giving feedback, and instead, you just come up with > > this flagrant nonsense about not being able to put up things on Maven > > Central (of course we can! LOL) .... or how it is so controversial that > the > > more advanced version of the codebase actually is more advanced (Of > course > > it is! LOL) .... or that some links being broken or the documentation > being > > patchy is somehow a permanent state of affairs... (Of course it's not!) > > > > Well, anyway, I felt I had to answer this, but if you spout more nonsense > > like this, I think I will just refrain from answering. In the past, I > have > > got into these annoying arguments with people because they got under my > > skin with this kind of stuff, but I suppose it's time to live and learn, > eh? > > > > Jon > > > > > > > > > > > >> - Ben > >> > >> Am Do., 9. Nov. 2023 um 18:40 Uhr schrieb Taher Alkhateeb > >> <ta...@pythys.com.invalid>: > >>> > >>> I'm a little confused. Why aren't we joining efforts on the apache > >> version? Why make it "a pity if a wider group of > >>> people never get the benefit of this work"? Am I missing something too > >> obvious or too old or something? Is this code base completely > incompatible? > >> Is this a technical issue? > >>> Taher Alkhateeb > >>> > >>> On Wednesday, November 08, 2023 04:03 +03, Jonathan Revusky < > >> revu...@gmail.com> wrote: > >>> Greetings, > >>> > >>> I thought to let people know that there is a vastly more advanced > version > >>> of FreeMarker available here:https://github.com/freemarker/freemarker3 > >>> > >>> You can build it via: > >>> > >>> git clonehttps://github.com/freemarker/freemarker3.git > >>> cd freemarker3 > >>> ant > >>> > >>> Or, if you want, there is a prebuilt jarfile you can grab here: > >>> https://parsers.org/download/freemarker.jar > >>> > >>> Though it is actually a rather superficial new feature, I think that > one > >>> thing that people will enjoy is the new terser syntax. Basically, if a > >>> directive starts a line (aside from whitespace) there is no need for > any > >>> pointy (or square) brackets. So you can just write: > >>> > >>> #if foo == bar > >>> blah blah > >>> /#if > >>> > >>> You can look here for a more complete description: > >>> https://github.com/freemarker/freemarker3/wiki/Terse-Syntax and here > is > >> an > >>> example of a template from the old test suite rewritten using the > terser > >>> syntax: > >>> > >> > https://github.com/freemarker/freemarker3/blob/master/src/freemarker/testcase/template/test-switch.html > >>> In this version of FreeMarker, the #assign and #local directives > (though > >>> they still work in a backward-compatible mode) were replaced with the > >> newer > >>> #var and #set. This is (IMHO) a significant improvement and is > described > >>> here:https://github.com/freemarker/freemarker3/wiki/Strict-Vars > >>> > >>> Just generally speaking though, the biggest changes are really under > the > >>> hood and would not be so visible to the casual user. This FreeMarker > >>> codebase has been refactored so that it largely does away with all of > >> those > >>> TemplateXXXModel wrappers and mostly just directly uses POJOs. (Plain > Old > >>> Java Objects.) This is described here: > >>> https://github.com/freemarker/freemarker3/wiki/Under-the-Hood > >>> > >>> Various longstanding annoyances, like not being able to directly use a > >> map > >>> with non-string keys, have been addressed. > >>> > >>> Oh, it suddenly occurs to me that many (perhaps most) people on this > >>> mailing list do not know who I am. I am effectively the original author > >> of > >>> FreeMarker. I say "effectively" because there was a FreeMarker 1.x, > which > >>> was really little more than a weekend hack. The version that 99% of > >>> FreeMarker users have used, which is 2.x, was a complete rewrite and is > >>> largely my work. > >>> > >>> As for other questions about what is going on with this, for example, > >> why I > >>> have put some renewed effort into FreeMarker after all years... well, > my > >>> main open source efforts have been going into my rewrite of that old > >> JavaCC > >>> parser generator that FreeMarker 2.x was originally built with. The new > >>> version of JavaCC was originally called FreeCC, then when I > resuscitated > >> it > >>> a few years ago, I called it JavaCC 21, but it is now rebranded as > >> CongoCC. > >>> So, since FreeMarker is a key part of CongoCC, I found myself adding > the > >>> occasional new feature to FreeMarker (my own version, not Apache > >>> FreeMarker). For example, the feature described here > >>> https://github.com/freemarker/freemarker3/wiki/Macros-as-Functions > was > >>> added to support CongoCC development back in 2020, but probably a lot > of > >>> FreeMarker users would appreciate this. > >>> > >>> So, at some point, I did rework FreeMarker to use CongoCC instead of > the > >>> legacy JavaCC. CongoCC is a much, much more powerful parser generator > >> than > >>> the original JavaCC, so it makes FreeMarker development comparatively a > >>> breeze. For example, I quite recently implemented assertions in > >> FreeMarker > >>> and this is where it is implemented: > >>> > >> > https://github.com/freemarker/freemarker3/blob/master/src/parser/Directives.inc.ccc#L417-L445 > >>> Or here is where ternary expressions are implemented: > >>> > >> > https://github.com/freemarker/freemarker3/blob/master/src/parser/Expressions.inc.ccc#L98-L118 > >>> You really should compare the FreeMarker grammar expressed with CongoCC > >> to > >>> the one that was written with legacy JavaCC, that is here: > >>> > https://github.com/apache/freemarker/blob/2.3-gae/src/main/javacc/FTL.jj > >>> > >>> So I rewrote FreeMarker (it is largely a rewrite at this point) to: (a) > >>> have a better tool for CongoCC development and (b) to provide a > showcase > >>> for CongoCC's capabilities. > >>> > >>> As for my plans, well, I do think it would be a pity if a wider group > of > >>> people never get the benefit of this work. Whether I intend to call > this > >>> version of FreeMarker "FreeMarker 3" or rename it to "Congo > Templates", I > >>> still haven't decided about that. I really only put some serious effort > >>> into the FreeMarker codebase starting this summer and the work kind of > >> took > >>> on a life of its own. > >>> > >>> In any case, anybody who is interested in getting involved, by all > means. > >>> Maybe start a discussion here: > >>> https://github.com/freemarker/freemarker3/discussions > >>> > >>> Best Regards and Greetings from Spain, > >>> > >>> Jonathan Revusky
