Since then I have also made a change that ensures that if the lambda
argument is null (which in FTL is the same as if the variable isn't
there at all), then it will not fall back to find an identically named
variable in higher variable scopes. This is important when doing
things like:
<#-- filters out null-s -->
myList?filter(it -> it??)
because if some day someone adds a variable called "it" to the
data-model, then suddenly the above won't filter out the null-s.
The same thing was always an issue with #list loop variables as well,
also with #nested arguments. So I have added a configuration setting
called "fallbackOnNullLoopVariable", which is by default true
(unfortunate historical baggage... but we can't break backward
compatibility). If you set it to false, then this will print "N/A" at
null list items, rather than "Clashing variable in higher scope":
<#assign it = "Clashing variable in higher scope">
<#list myList as it>
${it!'N/A'}
</#list>
These changes are pushed and deployed to the Apache snapshot Maven
repo in both branches.
So, apart from documentation, the local lambda feature is about ready,
or so I hope. I'm worried of rough edges though, so I think I will add
lambda support to some more builtins (?seq_contains, ?sort_by), and
explore some more use cases... If you have your own that you actually
keep running into, or want to be in the 2.3.29, tell it.
Monday, June 24, 2019, 1:59:21 AM, Daniel Dekany wrote:
> Well, I'm not exactly fast nowadays either... Anyway, I have pushed
> and deployed to the snapshot repo the changes I was talking about
> recently. That is, ?map or ?filter won't make a sequence out of an
> enumerable non-sequence (typically an Iterator) anymore. Because, it
> was the concern that if hugeResultSet is an Iterator because it's
> huge, then someone might writes:
>
> <#assign transformed = hugeResultSet?map(it -> something(it))>
> <#list transformed as it>
>
> instead of just
>
> <#list hugeResultSet?map(it -> something(it)) as it>
>
> and thus consuming a lot of memory without realizing it. So now if
> hugeResultSet wasn't already a sequence (List-like), the assignment
> will be an error, since we can't safely store a lazily transformed
> collection (lambdas will break), and we can't condense it down to a
> sequence (List-like thing) automatically either, as that might
> consumes too much memory. If hugeResultSet was a sequence, then it's
> not an error, as we assume that keeping all of it in memory is fine,
> as the original was stored there as well (in practice, most of the
> times... in principle we can't know).
>
> Now if the user feels confident about it, they can still write:
>
> <#assign transformed = hugeResultSet?map(it -> something(it))?sequence>
>
> Similarly, hugeResultSet?map(it -> something(it))[index] will be an
> error, as [index] is for sequences only, and ?map will not change a
> non-sequence to a sequence anymore. Similarly, if the user feels
> confident about it, they can write hugeResultSet?map(it ->
> something(it))?sequence[index].
>
> An interesting consequence of these is that ?sequence is now a bit
> smarter than before. Like if you write myIterator?sequnce[n], it will
> not fetch the elements into an in-memory sequence, it just skips n
> elements from myIterators, and returns the nth one. Similarly,
> myIterator?sequence?size won't store the elements in memory, it just
> counts them.
>
> As an interesting note, these two are also identically efficient:
>
> <#assign seq = hugeResultSet?filter(it -> something(it))?sequence>
> <#assign seq = hugeResultSet?sequence?filter(it -> something(it))>
>
> In both cases the actual conversion to a sequence (in-memory list)
> happens only just before assigning the value to seq. Once again,
> ?sequence now just means "it's OK to treat this as a sequence, however
> inefficient it is", and not "convert it to sequence right now".
>
>
> Friday, June 7, 2019, 10:38:50 AM, Christoph Rüger wrote:
>
>> These optimisations sound great. I will try to run some tests within the
>> next weeks. A bit busy lately.
>> Thanks
>> Christoph
>>
>> Am Mi., 29. Mai 2019 um 23:55 Uhr schrieb Daniel Dekany <ddek...@apache.org
>>>:
>>
>>> Tuesday, April 2, 2019, 12:10:16 PM, Christoph Rüger wrote:
>>>
>>> [snip]
>>> >> Well, if you fear users jumping on ?filter/?map outside #list for no
>>> >> good enough reason, there can be some option to handle that. But I
>>> >> don't think restricting the usage to #list is a good compromise as the
>>> >> default.
>>> >
>>> > I agree. Just keep as it is.
>>> >
>>> >> >> I'm not sure how efficiently could a configuration setting catch
>>> these
>>> >> >> cases, or if it should be addressed on that level.
>>> >> >
>>> >> > Maybe let's postpone configurability discussion a bit until the above
>>> is
>>> >> > more clear.
>>> >>
>>> >> In the light of the above, I think we can start thinking about that
>>> >> now.
>>> >
>>> > On that note on configurability: Would it be possible to programmatically
>>> > influence the Collection (Sequence) which is created under the hood?
>>> > E.g. by specifying a Factory? I ask because we are using something like
>>> > this (
>>> >
>>> https://dzone.com/articles/a-filebasedcollection-in-java-for-big-collections
>>> )
>>> > in other places for large collections. I know it is very specific, but
>>> just
>>> > wanted to bring it up.
>>> [snip]
>>>
>>> I think a good approach would be to ban the *implicit* collection of
>>> the result, when the filtered/mapped source is an Iterator, or other
>>> similar stream-like object that's often used for enumerating a huge
>>> number of elements. So for example, let's say you have this:
>>>
>>> <#assign xs2 = xs?filter(f)>
>>>
>>> If xs is List-like, then this will work. Since the xs List fits into
>>> the memory (although a List can be backed by disk, that's rather
>>> rare), hopefully it's not the kind of data amount that can't fit into
>>> the memory again (as xs2). On the other hand, if xs is an
>>> Iterator-like object, then the above statement fails, with the hint
>>> that xs?filter(f)?sequence would work, but might consumes a lot of
>>> memory.
>>>
>>> This is also consistent with how xs[i] works in the existing
>>> FreeMarker versions. That only works if xs is List-like (an FTL
>>> sequence). While xs[i] would be trivial to implement even if xs is
>>> Iterator-like, we don't do that as it's not efficient for a high i,
>>> and so the template author is probably not meant to do that. If he
>>> knows what's he doing though, he can write xs?sequence[i]. Yes, that's
>>> very inefficient if you only use [] once on that sequence, but you see
>>> the logic. map/filter breaks it, as xs?filter(f)[i] works even if xs
>>> is an Iterator, because filter/map currently always returns a
>>> sequence. If xs is Iteartor-like, then I want filter/map to return an
>>> Iterator-like as well, so then [] will fail on it.
>>>
>>> As a side note, I will make ?sequence smarter too, so that
>>> xs?sequence[i] won't actually build a sequence if xs is Iterator-like.
>>> It just have to skip the first i elements after all. (The ?sequence is
>>> still required there. It basically says: "I know what I'm doing, treat
>>> this as a sequence.")
>>>
>>> --
>>> Thanks,
>>> Daniel Dekany
>>>
>>>
>>
>
--
Thanks,
Daniel Dekany
