Also got some interesting test errors https://travis-ci.org/github/apache/freemarker-generator <https://travis-ci.org/github/apache/freemarker-generator>
Currently looking into it Siegfried > On 10.08.2020, at 20:53, Siegfried Goeschl <siegfried.goes...@gmail.com> > wrote: > > Hi Daniel, > > Please see my comments below > > Thanks in advance, > > Siegfried Goeschl > >> On 09.08.2020, at 22:00, Daniel Dekany <daniel.dek...@gmail.com >> <mailto:daniel.dek...@gmail.com>> wrote: >> >> See my answers inline... >> >> On Sun, Aug 9, 2020 at 7:16 PM Siegfried Goeschl < >> siegfried.goes...@gmail.com <mailto:siegfried.goes...@gmail.com>> wrote: >> >>> Hi Daniel, >>> >>> I think there is a way to define a custom FreeMarker template path in >>> IntelliJ - see >>> https://intellij-support.jetbrains.com/hc/en-us/community/posts/115000686590-How-to-configure-the-freemarker-template-path-or-what-s-the-default-path- >>> >>> <https://intellij-support.jetbrains.com/hc/en-us/community/posts/115000686590-How-to-configure-the-freemarker-template-path-or-what-s-the-default-path-> >>> >>> Having said that we should take care that the IntelliJ plugin is happy >>> >>> * Some users do create web applications based on Apache FreeMarker >>> * Having FTL support built-in is a huge bonus for beginners >>> >> >> What do you mean, realistically what can we do, in case that plugin is >> indeed that broken? I mean, for FreeMarker Generator alone it doesn't >> really matter for users. For all the other projects, FreeMarker doesn't >> prevent them from doing dirty things to accomodate to the plugin (as you >> saw yourself ;) ), and I'm not sure what else could FreeMarker itself do >> about this. Fixing/extending the plugin itself is also not likely, as it's >> closed source. (Of course, it would be great to have an open source LSP >> language server, and then adding open source plugins based on that... but >> things go very, very slowly around here.) > > Played around with the IntelliJ plugin but I don't get it perfectly working > ... >> >> Regarding the intended role of "/templates" >>> >>> * The implemented template loading tries to resolve the given template >>> name as file and if there is no file the actual template loading is >>> delegated to a "MultiTemplateLoader" >>> * In https://issues.apache.org/jira/browse/FREEMARKER-146 >>> <https://issues.apache.org/jira/browse/FREEMARKER-146> I split the >>> existing templates into a reusable part (living now in "templates") and >>> examples (living now in "examples") >>> * IMHO the "templates" directory should contain commonly useful templates >>> - so they could be considered as guaranteed functionality >>> * Transforming CSVs >>> * Transforming Excel files to CSV, HTML & Markdown (and maybe >>> Confluence Markup) >>> * Transform JSON to YAML and the other way around >>> * I'm happy to have those templates as files in the installation >>> directories - better visibility and user can add there own templates (if >>> the want to play nasty) >>> >> >> The fundamental problem with that is this. Currently, if you pull in >> freemarker-generator-cli as Maven dependency, the templates will not be >> available. Surely, because it's the CLI, you could say that it's not >> supposed to be used without the FreeMarker Generator Home Directory created >> somewhere, which contains the launch script and templates/ and all. But, if >> these templates are guaranteed functionality in FreeMarker Generator, then >> they don't strictly belong to the CLI. When we will have a proper Maven >> plugin for example, they should be still accessible. In that case you only >> have your Maven dependencies, so the templates must come from there. >> >> Regarding visibility, it's a bit like with Java. Java classes are not too >> readable without looking at the source code either. That's not an advantage >> when it comes to "visibility", sure. But luckily this is open source, and >> it's very easy to get to the source code, if someone really cares (like >> from the Maven source artifact). That applies to core stuff implemented in >> FTL as well. So, the previously mentioned advantage (that they are >> available from a plain dependency) certainly overweights this disadvantage >> (less visibility). >> >> I don't think we want users to add their own templates to /templates, as >> they will be potentially lost when someone updates FreeMarker Generator, >> and is confusing for colleagues, etc. If we can prevent bad practices, >> let's do it. Especially as we also provide a cleaner >> solution, ~/.freemarker-generator/ added to the template root (which will >> change to ~/.freemarker-generator/templates now, as you say below). If >> people need a system-wide one, we can also add /etc/freemarker-generator, >> or something like that. >> > > I currently disagree here - I like the visibility aspect and it is pretty > difficult to get rid of templates loaded from the classpath. > >> What I want to change >>> >>> * Use "templates" as FreeMarker template root for the CLI installation and >>> "~/.freemarker-generator" directory >>> * Adopt the examples and documentation, e.g. "freemarker-generator -t >>> info.ftl" instead of "freemarker-generator -t templates/info.ftl" >>> >> >> Except, we should avoid adding info.ftl and such to the template directory >> root. Not only because of potential name clashes, but because it can be >> confusing if someone looks at that (where's info.ftl coming from). How >> about reserving the "freemarker-generator" directory on the top level, and >> then it's "freemarker-generator -t freemarker-generator/info.ftl". >> (Hopefully that helps realizing that it's something built into FreeMarker >> Generator.) Actually, maybe it should be even >> "freemarker-generator:/info.ftl ". > > Mhmm, already pushed the changes :-( > > Need to think about it > >> >> Thanks! >> Daniel Dekany >> >> >>> What do you think? >>> >>> Siegfried Goeschl >>> >>> >>>> On 05.08.2020, at 13:28, Daniel Dekany <daniel.dek...@gmail.com >>>> <mailto:daniel.dek...@gmail.com>> wrote: >>>> >>>> Ah, you have the FreeMarker plugin in IntelliJ, and that assumes >>> something >>>> crazy like that? The Community Edition doesn't have that plugin, and I >>>> never saw it. But if that plugin indeed assumes that the template root is >>>> the project root... that wouldn't make sense, in any real world project. >>>> Even if some have a "templates" directory laying around in the directory >>> as >>>> the POM, then that would be the template root (means, a template path is >>>> like "/foo.ftl", not "/templates/foo.ftl"), not the whole project. So, I >>>> certainly don't think we should align with a broken plugin, especially as >>>> most users won't be affected (they just use the binary). >>>> >>>> Also, I'm a bit confused about the intended role of "/templates". Is it >>>> core (guaranteed) functionality in Generator CLI? As opposed to a core >>>> functionality of Generator itself? (Sure, for now Generator can be called >>>> via CLI only, but you see what I mean.) Or is it just some examples >>> laying >>>> around, whose presence users shouldn't count on in their projects? If >>> it's >>>> core functionality, then it certainly should be a Java resource, packed >>>> into the jar. >>>> >>>> Also let's not skip the question from the earlier mail, regarding what >>> path >>>> should this be available for the TemplateLoader (and for -t)? >>>> >>>> On Wed, Aug 5, 2020 at 11:47 AM Siegfried Goeschl < >>>> siegfried.goes...@gmail.com <mailto:siegfried.goes...@gmail.com>> wrote: >>>> >>>>> Hi Daniel, >>>>> >>>>> When moving the "templates" to a subdirectory IntelliJ complains about >>> not >>>>> finding the FTL include files - technically it is working but without >>>>> tweaking IntelliJ it spits out errors. And the Travis tests failed >>> while my >>>>> local tests succeeded - see >>>>> >>> https://travis-ci.org/github/apache/freemarker-generator/builds/714600403 >>> <https://travis-ci.org/github/apache/freemarker-generator/builds/714600403>. >>>>> >>>>> Regarding template root directory - I guess there is some room for >>>>> improvements >>>>> >>>>> * I'm passing a list of template root directories (current directory, >>> CLI >>>>> installation directory, ~/.freemarker-cli/" >>>>> * The "templates" directory is just the way I organised the overall file >>>>> structure >>>>> * Additional template root directory can be passed on the command line >>>>> * The implementation can pick up any FTL file using absolute or relative >>>>> file paths (and URLs) >>>>> >>>>> Thanks in advance, >>>>> >>>>> Siegfried Goeschl >>>>> >>>>> >>>>>> On 04.08.2020, at 22:12, Daniel Dekany <daniel.dek...@gmail.com >>>>>> <mailto:daniel.dek...@gmail.com>> >>> wrote: >>>>>> >>>>>> Can you clarify, what part of IntelliJ is happy about that, and why? >>>>>> >>>>>> Also it reminds me of another thing I wrote about earlier. Generally, >>> we >>>>>> say that the template root directory shouldn't not contain anything but >>>>>> templates, so if the project root is the template root directory, or >>> the >>>>>> freemarker-generator home directory, that's a bit fishy. Yes, the >>> reason >>>>>> for this purist approach is partially security concerns with web >>>>>> applications, and this is not one. But whatever the original reasons >>> are, >>>>>> FreeMarker has this omnipresent concept of the template root directory, >>>>>> which is kind of a virtual root directory for a template (i.e., they >>> can >>>>>> leave it, and it's all templates, or rarely files referred from >>>>> templates, >>>>>> like images). >>>>>> >>>>>> Also, it's strange/confusing if the template root directory has a >>>>>> "templates" subdirectory. I mean, the template root directory is what >>> one >>>>>> would normally call the templates directory. As a template author, I >>>>> would >>>>>> expect some path like "/freemarker-generator/whatever.ftl" (also note >>>>> that >>>>>> it's an absolute path). There, "/freemarker-generator/" tells me that >>>>> this >>>>>> is something included in freemarker-generator, and not templates of my >>>>>> project. >>>>>> >>>>>> On Tue, Aug 4, 2020 at 8:51 PM Siegfried Goeschl < >>>>>> siegfried.goes...@gmail.com <mailto:siegfried.goes...@gmail.com>> wrote: >>>>>> >>>>>>> I'm not happy about it :-) >>>>>>> >>>>>>> * A top-level "templates" directory makes IntelliJ happy when includes >>>>> are >>>>>>> used >>>>>>> * I have an ExamplesTest which tests each documented command line >>> usage >>>>>>> and I prefer to have it in sync with the documentation >>>>>>> * I actually tried it but reverted the changes since the Travis build >>>>> fails >>>>>>> >>>>>>> Thanks in advance, >>>>>>> >>>>>>> Siegfried Goeschl >>>>>>> >>>>>>> >>>>>>>> On 28.07.2020, at 22:34, Daniel Dekany <daniel.dek...@gmail.com >>>>>>>> <mailto:daniel.dek...@gmail.com>> >>>>> wrote: >>>>>>>> >>>>>>>> It's just that "templates" belongs to the "main" source code of the >>>>>>>> freemarker-generator-cli, similarly to >>>>> config/freemarker-cli.properties, >>>>>>>> which is under src/main. (I think many of us expect a Maven project >>>>> root >>>>>>>> directory to only contain the "noise" needed for build and legal, and >>>>>>> "src" >>>>>>>> that contains *all* the essential source code.) Also, as the source >>>>> tree >>>>>>>> doesn't mirror the binary distribution directory tree anyway, so I'm >>>>> not >>>>>>>> sure if there's any fundamental reason to make an exception with >>>>>>>> "templates", and put it under the root. >>>>>>>> >>>>>>>> On Tue, Jul 28, 2020 at 4:09 PM Siegfried Goeschl < >>>>>>>> siegfried.goes...@gmail.com <mailto:siegfried.goes...@gmail.com>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi folks, >>>>>>>>> >>>>>>>>> Back from the mountains and in front of the keyboard again :-O >>>>>>>>> >>>>>>>>> * I created a JIRA ticket and will push a feature branch soon (bad >>>>>>> habits >>>>>>>>> die hard) >>>>>>>>> * I will go through the licences (review and collect in "licenses" >>>>>>>>> directory) >>>>>>>>> * Good point about plugin versions being defined in apache POM - >>> will >>>>>>>>> rework them >>>>>>>>> * I will delete the existing configuration of the "release" profile >>>>>>>>> >>>>>>>>> Some things to discuss >>>>>>>>> >>>>>>>>> * What are the benefits to move "templates" to "src/main/templates"? >>>>>>>>> Currently they are in sync with "freemarker-cli" which is quite nice >>>>> for >>>>>>>>> tests & documentation ... >>>>>>>>> >>>>>>>>> Thanks in advance, >>>>>>>>> >>>>>>>>> Siegfried Goeschl >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 26.07.2020, at 01:27, Daniel Dekany <daniel.dek...@gmail.com >>>>>>>>>> <mailto:daniel.dek...@gmail.com>> >>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> I said I will help in the Apache release process, so only focusing >>> on >>>>>>>>> that, >>>>>>>>>> so some points: >>>>>>>>>> >>>>>>>>>> - We are required to have a so-called source release (every other >>>>>>>>>> artifact is optional in the policy). As we are using the >>>>>>>>> org.apache:apache >>>>>>>>>> parent, that should generate that automatically, with .asc and >>> sha512 >>>>>>>>> and >>>>>>>>>> all. But currently it doesn't, because maven-release-plugin >>>>>>>>> config/argument >>>>>>>>>> is overwritten with this: >>>>>>>>> <arguments>-Dmaven.javadoc.skip=true</arguments>. >>>>>>>>>> We should keep configuring release at minimum, to avoid such >>>>>>> accidents. >>>>>>>>>> Maybe as in >>>>>>>>>> >>> https://github.com/apache/freemarker-docgen/blob/master/pom.xml#L70 >>> <https://github.com/apache/freemarker-docgen/blob/master/pom.xml#L70>. >>>>>>>>>> - I assume we also want a binary release, for the CLI only, and >>>>>>>>>> freemarker-generator-cli-x.y.z-*app*.zip (note the "-app") will be >>>>> our >>>>>>>>>> binary release artifact. Then: >>>>>>>>>> - It bundles some dependency binaries that are not under ASL2 >>>>> license. >>>>>>>>>> Unfortunately, the licenses of those must be included in the >>>>>>>>>> distribution. >>>>>>>>>> See the LICENSE at >>>>>>>>>> https://github.com/apache/freemarker-docgen/blob/master/LICENSE >>>>>>>>>> <https://github.com/apache/freemarker-docgen/blob/master/LICENSE>. >>>>>>> At >>>>>>>>>> the bottom, it lists the licenses, then it refers to the actual >>>>>>>>> license >>>>>>>>>> files. As we will have many licenses, let's create a "licenses" >>>>>>>>> directory >>>>>>>>>> for them. (In the future, the dependencies have to be checked >>>>>>>>>> for changes. >>>>>>>>>> Even version upgrades my pull in sneaky transient dependencies. >>>>>>> Some >>>>>>>>>> licenses are not even allowed, so anything but ASL2, MIT, >>>>>>>>>> BSD-without-advertisement-clause, will need closer attention.) >>>>>>>>>> - I noticed that the documentation is not included in the binary >>>>>>>>>> distribution. But because of the extra legal burden including it >>>>>>>>> would >>>>>>>>>> bring (we have fonts and icons under CC-SA and SIL OFL in the >>>>>>> Docgen >>>>>>>>>> output), I actually prefer that to stay like that. >>>>>>>>>> - .sha512 file is not yet generated >>>>>>>>>> - freemarker-generator-cli/src/site: If you agree, instead of this >>> I >>>>>>>>>> will create freemarker-generator*-site*/src/docgen, and convert the >>>>>>>>>> Markdown to XDocBook. For now this will be only the CLI >>>>> documentation, >>>>>>>>> and >>>>>>>>>> the JavaDoc, as the freemarker-generator-maven-plugin is not ready. >>>>>>> One >>>>>>>>>> annoyance I realized is that we should have Docgen in Maven Central >>>>>>>>> for the >>>>>>>>>> builds to work reliably in the future, which means that Docgen has >>> to >>>>>>>>> be >>>>>>>>>> officially released (it never was, it's an internal tool). That >>> would >>>>>>>>> be a >>>>>>>>>> minimalistic release, means, no announcement, no web site, just the >>>>>>>>> bare >>>>>>>>>> minimum (i.e., source release, and deployment to Maven Central). I >>>>>>> have >>>>>>>>>> some backlog there (Google keeps nagging me about mobile issues), >>> but >>>>>>> I >>>>>>>>>> hope I can fix that in the coming days, then go through the >>> official >>>>>>>>>> release process (takes 1-2 weeks). >>>>>>>>>> - Some smaller things: >>>>>>>>>> - >>>>>>>>>> - Having a "release" profile is also hopefully unnecessary, >>>>> because >>>>>>>>>> org.apache:apache takes care of signing. >>>>>>>>>> - We should also remove most plugin version management, as many >>> of >>>>>>>>>> those versions are set in org.apache:apache. >>>>>>>>>> - freemarker-generator-cli/templates should be inside >>>>>>>>>> freemarker-generator-cli/src/main/templates, I guess. >>>>>>>>>> >>>>>>>>>> P.s.: Siegfired asked our opinions in another thread. I did my >>> part, >>>>>>> even >>>>>>>>>> too much (;, so, would be good if others participate in that as >>> well. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Best regards, >>>>>>>>>> Daniel Dekany >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Best regards, >>>>>>>> Daniel Dekany >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> Best regards, >>>>>> Daniel Dekany >>>>> >>>>> >>>> >>>> -- >>>> Best regards, >>>> Daniel Dekany >>> >>> >> >> -- >> Best regards, >> Daniel Dekany
