I don't see how exposing DataSources in itself is sufficient. Because, in the template you primarily just need a list of non-named data sources, and then a map of named data sources. One only contains the data sources passed by position, and the other only contains the data sources passed by name. If we agree on that, then the next question is naming. It could be dataSources[indexExpression], and namedDataSource.someName (namedDataSources[nameExpression]). However, it is probably worth it to pull some more TemplateModel tricks here, and make dataSources a sequence+hash type value, so we can get away with a single variable. Then you can write dataSources[indexExpression], dataSources.someName, and dataSources[nameExpression]. To access advanced functions, like the find method, you may expose the DataSources object as is, under a different name. Like tools.dataSources, or such.
As for what to do when you have exactly one data source, yes, there could be a dataSource variable (singular) that only works when you have exactly one positionally passed data source. However, for usability reasons, that variable should always exist, so if somebody tries to use it when they shouldn't, it can give a helpful error message. .dataSource: For that it had to be part of the template language, so although our unique position makes that achievable, probably not. Especially as it doesn't really solve the problem, since you have other variables as well. Like "tools". The best we can do is to keep the number of such variables low. One of my statements was that using URL-s (or filenames) as keys is not useful for the template author. If the user didn't specify a name for the data source, then they shouldn't be able to access it by name either, as there's no stable name to use. What about that? On Thu, Sep 30, 2021 at 10:43 PM Siegfried Goeschl <siegfried.goes...@gmail.com> wrote: > > Hi Daniel, > > To recap the usage of DataSources ... > > * A template does not use any data source, e.g. copying & expanding based on > environment variables > * A template uses exactly one data source, e.g. some CSV to HTML > transformation > * A template can use one or more named data sources, e.g. > "users=https://some.url <https://some.url/>" > * A template aggregating a list of unnamed data sources, e.g parsing and > extracting stuff from access logs > > Over the time I implemented various approaches > > * Having a plain list of data sources > * Wrapping the data sources in custom holder - see > https://github.com/apache/freemarker-generator/blob/master/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/datasource/DataSources.java > > <https://github.com/apache/freemarker-generator/blob/master/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/datasource/DataSources.java> > * Later one we migrated to a plain map forcing requiring unique keys (so I am > using HTTP URIs here) > > So what about the following approach (you suggested something like this some > time ago) > > * Exposing a "dataSource" when exactly one data source is provided > * Exposing the org.apache.freemarker.generator.base.datasource.DataSources as > "dataSources" again? It is not super nice in FreeMarker terms but provides > convince methods to slice through data sources > > Related question - to avoid name clashes with user variables - is it possible > to use reserved names as well > > * ${.dataSource} > * ${.dataSources} > > What do you think?! > > Siegfried > > > On 28.09.2021, at 13:01, Daniel Dekany <daniel.dek...@gmail.com> wrote: > > > > Yeah, I did not answer that, but I did check the feature. It allows > > using the project in more use-cases, so it's definitely an > > improvement. > > > > That said, my main pain point remains the way data sources are handled > > (that's regardless of this feature). > > Like having dataSources?values[0] all over the place is not nice, > > given it's quite central feature of the project. And that thing is > > like that because dataSources is a Map where the key is the source > > URL. But when would someone want to get something by URL? And if > > somebody has that idea, I would be against it in most cases, as it's > > fragile (like I would totally pass in a file with a different path or > > name, believing that doesn't matter). Instead, I think: > > - dataSources should be just a list of objects. (From that object you > > can then get file name etc., though normally you shouldn't care about > > it.) It should only contain the data sources for which no name was > > explicitly specified by the user. > > - There should be a separate namedDataSources, which only contains the > > data sources that the user has explicitly given a name > > > > I remember this was a topic back then. I don't know, maybe revisit it. > > I think it would be much less confusing like above. > > > > > > On Sat, Sep 11, 2021 at 3:55 PM Siegfried Goeschl > > <siegfried.goes...@gmail.com> wrote: > >> > >> Worx now - thx > >> > >> Thanks in advance, > >> > >> Siegfried Goeschl > >> > >> > >>> On 11.09.2021, at 11:38, Daniel Dekany <daniel.dek...@gmail.com> wrote: > >>> > >>> I hopefully fixed that build issue now. > >>> > >>> On Sat, Sep 11, 2021 at 9:20 AM Siegfried Goeschl < > >>> siegfried.goes...@gmail.com> wrote: > >>> > >>>> Hi Daniel, > >>>> > >>>> Had a quick look at FREEMARKER-154 but document generation fails for me > >>>> but I have no time to look into it > >>>> > >>>> [INFO] --- freemarker-docgen-maven:0.0.2-SNAPSHOT:transform > >>>> (docgen-transform) @ freemarker-generator-website --- > >>>> Using FreeMarker 2.3.31 > >>>> Loading > >>>> /Users/sgoeschl/work/github/apache/freemarker-generator/freemarker-generator-website/src/main/docgen/book.xml... > >>>> Output directory: > >>>> /Users/sgoeschl/work/github/apache/freemarker-generator/freemarker-generator-website/target/website > >>>> [INFO] > >>>> ------------------------------------------------------------------------ > >>>> [INFO] BUILD FAILURE > >>>> [INFO] > >>>> ------------------------------------------------------------------------ > >>>> [INFO] Total time: 2.237 s > >>>> [INFO] Finished at: 2021-09-11T08:53:17+02:00 > >>>> [INFO] > >>>> ------------------------------------------------------------------------ > >>>> [ERROR] Failed to execute goal > >>>> org.apache.freemarker.docgen:freemarker-docgen-maven:0.0.2-SNAPSHOT:transform > >>>> (docgen-transform) on project freemarker-generator-website: Error during > >>>> document transformation: Insertable file with symbolic name > >>>> "websitePomGenerated" points to a directory that doesn't exist: > >>>> "/Users/sgoeschl/work/github/apache/freemarker-generator/freemarker-generator-website/target/docgen-insertable-outputs" > >>>> -> [Help 1] > >>>> > >>>> > >>>> And XMLMind looks quite nice to use ... > >>>> > >>>> Skimmed through comments regarding to documentation > >>>> > >>>> * I used the examples in the documentation as mental notes so many of > >>>> them might be irrelevant and should be removed altogether > >>>> * And yes, a lot of stuff needs to be rewritten .-) > >>>> * Gomplate documentation is under MIT licence > >>>> * Since I don't have the final output - how annoying are all the ASL > >>>> headers in the sample file? Guess they add a lot of noise? > >>>> > >>>> So I suggest to get FREEMARKER-188 out of the door since this is the > >>>> biggest pain point regarding initial release and move the documentation > >>>> step by step - what do you think? > >>>> > >>>> Thanks in advance, > >>>> > >>>> Siegfried Goeschl > >>>> > >>>> > >>>> > >>>>> On 09.09.2021, at 21:06, Daniel Dekany <daniel.dek...@gmail.com> wrote: > >>>>> > >>>>> Maybe it's already too late, but I will... hopefully at the weekend. > >>>>> > >>>>> Oh, and the documentation conversion too... what was completed will drag > >>>>> behind the md-s again. /-: Sorry about that. Have you ever tried using > >>>> what > >>>>> we have in that branch? It can save time and improve the quality of the > >>>>> content, as it inserts the actual example files, runs them, and inserts > >>>> the > >>>>> actual output. No copy-pasting. So it's guaranteed that what we tell > >>>>> is accurate and up to date. XXE is another time saver for me... though > >>>>> admittedly it needs initial investment to use. (Don't even touch XML > >>>>> editors where you edit on source level... that's clearly not for > >>>>> documentation.) > >>>>> > >>>>> On Tue, Sep 7, 2021 at 9:38 PM Siegfried Goeschl < > >>>>> siegfried.goes...@gmail.com> wrote: > >>>>> > >>>>>> Hi folks, > >>>>>> > >>>>>> After Corona, lockdown, home schooling and endless telcos and finally > >>>>>> returned to my pet project :) > >>>>>> > >>>>>> I'm tackling now the age-old topic of generating output files driven by > >>>>>> data sources - https://issues.apache.org/jira/browse/FREEMARKER-188 < > >>>>>> https://issues.apache.org/jira/browse/FREEMARKER-188> > >>>>>> > >>>>>> Daniel, since we invested a lot of energy discussing the topic you are > >>>>>> happily invited to share your thought :) > >>>>>> > >>>>>> Thanks in advance, > >>>>>> > >>>>>> Siegfried Goeschl > >>>>>> > >>>>>> > >>>>> > >>>>> -- > >>>>> Best regards, > >>>>> Daniel Dekany > >>>> > >>>> > >>> > >>> -- > >>> Best regards, > >>> Daniel Dekany > >> > > > > > > -- > > Best regards, > > Daniel Dekany > -- Best regards, Daniel Dekany
