Find answers inline. On Sun, Jul 5, 2020 at 9:31 AM Siegfried Goeschl < [email protected]> wrote:
> Hi Daniel, > > thanks for your feedback ... > > # 1. Site Generation > > Maven is not soo bad - I uploaded the generated site to > http://www.senilesoftwareengineer.org/freemarker-generator/ - I > personally have no problem using FreeMarker documentation approach but > decided no to follow it up for the time being due to time constraints. > So if I convert the markdown to XDocBook, and set up site generation for it, and thereafter you will edit XDocBook (I recommend XXE for that), will that be OK with you? # 2. Design/Architectural Doubts > > ## 2.1 Parsed Data > > Currently we can load files/URLs as "DataSource" (which need to be > processed in FTL) or we can use "-m / --data-model" to parse and expose > the content directly (see > > http://www.senilesoftwareengineer.org/freemarker-generator/freemarker-generator-cli/cli/concepts/data-models.html > ). > (Yeah, that's where I said that it looks like a quick ad-hoc solution, and as such is quite limited.) Regarding a less programmatic way - Named URIs are used (see > > http://www.senilesoftwareengineer.org/freemarker-generator/freemarker-generator-cli/cli/concepts/named-uris.html) > > to pass additional metadata but the implementation requires some re-work > to pass arbitrary metadata in the fragment. > > E.g. what should be possible is > > ``` > freeMarker-cli -t some.ftl -m > > user=somepath/user.csv#delimiter=TAB&format=DEFAULT&charset=UTF-16&header=false > ${CSVtool.parse(DataSources["user"])} > ``` > The idea is to be able to pass all metadata being used to parse the data > source - that would be also helpful for defining a SQL data source. > Yes. But I guess we should break out from the preconception that a data source must be an URL, and also that therefore we have to encode all the parsing and selection parameters into the anchor as name-value pairs (which also implies that you might have % escapes). Because we sometimes List and Map<String, Object>-like data structures as well, etc. (Also, using the anchor is kind of an abuse of the URL syntax, and &name=value is query parameter syntax, not anchor syntax. So this is maybe confusing for users, like they will keep writing ? instead of #.) My idea is that a Data Source should be the name of a Data Loader (like, "csv", "excel", "jdbc", etc.), plus its parameters. To specify that, we should use FTL method call syntax, because that syntax the users have to learn anyway, and it's still relatively expressive (and can be re-used in actual templates - see that later): -s 'myTable=csv("foo/bar.csv", {"format": "EXCEL", "delimiter": ";"})' If we have this (which has basically the same meaning as it has currently): -s myRawTable=foo/bar.csv that could be just a shorthand for: -s myRawTable='raw("foo/bar.csv")' So "raw" data is what you pass to "tools" for parsing in the current version. If you prefer do parsing in the templates, then you can reuse what you learned about -s, as it's pretty much the same: <#assign myTable = DataLoaders.csv(DataSources.myRawTable, { "format": "EXCEL", "delimiter": ";" })> Also, if you decide to parse data inside the template, doing it will be very similar as in the -s switch. > ## Data versus Template Driven Generation > > Currently a variety of transformations are supported on the command line > (see > > http://www.senilesoftwareengineer.org/freemarker-generator/freemarker-generator-cli/cli/concepts/transformation.html > ) > > * single template > * multiple templates > * directory of templates > > Regarding "template versus data driven" generation - CLI does not > support data-driven generation where one output file is generated per > data source. > Yes. That's exactly what I feel is a major mistake on the long run. At least we should be able to add it later, but I guess, to ensure that, we pretty much have to implement it. So what will happen with this, what's your standpoint? Furthermore, I don't think that we should have a "mode" for this, which applies for a whole run. I think I'm repeating myself from some months ago, but, to recap, there are simply "seeds" that cause generating an output file. That "seed" can be a template, or a data file, or part of loaded data (like each row in a table is a "seed"). And I believe it's worth it to allow mixing these in the same run, because then freemarker-generator will be good enough fit for more use-cases. So for example, you have some JSON files in your project, each will generate a java file (typical source code generation), but you also have templates for pom.xml, etc., that will generate its own output. ## 2.2 Convergence of Maven Plugin & CLI > > It is on my list but does not have high priority since it should not > affect Maven plugin usage (regarding backward compatibility). > But that would be a completely new pugin, I assume. # 3. Maturity / Backward Compatibility > > The project is not mature - we have no real users and little field > testing - in short I don't want to guarantee backward-compatibility for > now because some of my decisions might turn out to be plain wrong (or > even stupid) :-) > > According to https://semver.org we can release 0.x.y and follow > https://semver.org/#spec-item-4 - not so nice but pragmatic. > > # 4. General Considerations > > There is a danger we never get "freemarker-generator" out of the door > ("perfect is the enemy of good") therefore the whole PMC should look > into the questions/answers * "freemarker-generator" needs community usage to become really useful - > the unreleased stuff does not add any value > * Is it acceptable to release a 0.X.Z where we not backward-compatible > until 1.0.0? > With the clear warning that we won't maintain this "branch" (a series of backward compatible releases), etc. we can release. That, however, discourages usage. And that's really the problem, not that we can't release, because we can publish stuff in whatever development stage (assuming users will be well informed). But if we are to hope for any significant user base, I believe we should be able to promise some stability, and maintenance of the BC branch. That's why I'm saying that we should get the core architecture about right, and the project scope. (I'm not talking about missing features that can be added later, nor about bugs and other rough edges.) > * Are there any deal breakers which needs to fixed before we start with > a first public release? > As far as I'm concerned (but I'm just one of the PMC members), if you have strong feelings for doing a release, with the proper warnings, sure, I will collaborate. The deal breaker is more for the users, who get no good BC promise or maintenance. Thanks in advance, > > Siegfried Goeschl > > On 4 Jul 2020, at 1:10, Daniel Dekany wrote: > > > It should be in https://freemarker.apache.org/generator/. You will > > need to > > commit/push into > > https://github.com/apache/freemarker-site/tree/asf-site > > for that. (See > > https://freemarker.apache.org/committer-howto.html#updating-homepage) > > > > We need to generate the web pages somehow though. I saw you try to do > > that > > with Maven "site", but personally, I can't imagine that it can be > > tweaked > > to generate reasonable output. Not sure how others see this. Maven > > "site" > > is basically the Maven model dumped into HTML pages, and "reports" > > like > > even the log of the Rat run etc. Dozens of menus and sub-pages for > > details > > users mostly don't care about. The actual content users do care about > > is > > stuffed under "About", and from there it's not properly navigable or > > searchable, as apparently it's assumed to be a single page. I know > > back > > then you didn't want to use the tool that's used both for the main web > > site > > and for the FreeMarker documentation, but I think that's the most > > economical solution currently (and then we also have common styling, > > and > > common place to fix whatever technical issues), so please reconsider > > that. > > (I can help setting that up, of course.) > > > > Now, some more difficult-to-address problematic things. My problem is > > that > > these just weren't concluded. Discussion just died. To be concluded > > can > > even mean that other PMC members say we should step over these, even > > if I > > disagree. But these should be understood, and considered. So, these > > are: > > > > - Design/architectural doubts that are probably not realistic to > > address > > much later: > > - Currently, data sources are just URL-s (locations), and > > templates > > are meant to call tool API-s to parse them. As I said back then, > > one > > consequence is that then, you can't put parsed data into the > > data-model > > shared by all templates (i.e., via -m/--data-model). Because, > > you have no > > convenient/concise way to load the actual (parsed) data, instead > > you have > > to "program it" in FTL, because the assumption was that you only > > want to do > > that in templates anyway. Furthermore, as I saw it just now, > > --data-model > > actually supports some ad-hoc way of parsing the data pointed by > > a > > data-source-like URL, as JSON, YAML, maybe some more. Here's an > > example of > > that: `-m config=env:///DB_CONFIG#mimetype=application/json`. So > > there is a > > need actually. But compared to what you can do in templeats, > > it's totally > > different, and of course very restrictive. I was also looking > > for a less > > "programmatic" way of loading data because even doing it in > > templates is > > not very friendly as it is. (For ultimate flexibility you might > > need > > program logic for sure, but certainly not just to grab a > > worksheet from an > > Excel file.) We also should have a POC for loading from a less > > file centric > > data source, like from a DB with a SQL query, to see if the > > concept works. > > - Not sure what's with template-driven VS data driven output > > generation. Data driven is when you generate one output file per > > some data > > unit (like per JSON file, or per DB record), and all is > > transformed by a > > common template. How would that look currently in > > freemarker-cli? > > - The Maven plugin provides very different functionality than the > > CLI. > > The original concept was that they are just two interfaces to the > > same > > product. That was also brought up, and I don't think it was > > addressed much, > > like, why you disagree, if you do, or what's going on. Anyway, > > regarding > > what we have now. The naming implies that, as both are "FreeMarker > > Generator", just one is "CLI", and the other is "Maven". But as far > > as I > > see, they are two different "products" really, focusing on > > different use > > cases. So, if it stays like this, then some of them have to be > > renamed at > > least. Or I don't know what others think we can do. (Then it's > > still > > somewhat weird to release them together.) > > > > (I have also run through the CLI documentation, and found a lot of > > less > > fundamental things to address, but I don't want to overload the > > thread.) > > > > Also, if the project is released mostly as it is now, what will we > > promise > > regarding backward compatibility? What do we communicate about the > > matureness of the project? > > > > On Wed, Jul 1, 2020 at 8:29 AM Siegfried Goeschl < > > [email protected]> wrote: > > > >> Hi folks, > >> > >> I'm pretty much finished with the stuff I would like to have for the > >> first > >> public release but there is a still a lot of work ahead > >> > >> * FREEMARKER-148 [freemarker-cli] Make usage of "DataSources" more > >> "Freemarker" like > >> * FREEMARKER-150 [freemarker-cli] Setup "freemarker-generator" web > >> site > >> * Setup the release process (Daniel, I guess I need some help here) > >> * An iteration of reviews and discussions > >> > >> Some thought along the line > >> > >> * I would like to finish FREEMARKER-148 this week > >> * Daniel, any suggestion where to host the public website? > >> > >> Thanks in advance, > >> > >> Siegfried Goeschl > >> > >> > > > > -- > > Best regards, > > Daniel Dekany > -- Best regards, Daniel Dekany
