Hi Fabian
So you're saying that with a windowed stream I can still emit a folded
aggregate for each event as it comes in? I didn't realize that, I
thought that windows was a sort of micro batching.
I'll go read the link you posted
Thanks
--
Bart van Deenen
bartvandee...@fastmail.fm
On Fri, Mar 18, 2016, at 11:54, Fabian Hueske wrote:
> Hi Bart,
> if you run a fold function on a keyed stream without a window, there
> is no way to remove the key and the folded value.
> You will eventually run out of memory if your key space is
> continuously growing.
>
> If you apply a fold function in a window on a keyed stream you can
> bound the "lifetime" of the key and value.
> Similar as with a non-windowed fold, you can emit a record for each
> incoming record. Additionally, you can register a timer to purge the
> window content after a certain time (such as a few days). This blog
> post should be a good introduction into Flink's window and trigger
> mechanism [1].
>
> Best, Fabian
>
> [1] http://flink.apache.org/news/2015/12/04/Introducing-windows.html
>
>
> 2016-03-18 11:42 GMT+01:00 Bart van Deenen <bartvandee...@fastmail.fm>:
>> If I do a fold on a KeyedStream, I aggregate events for such-and-such
>>
key.
>>
My question is, what happens with the aggregate (and its key) when
>>
events for this key stop coming?
>>
My keys are browser session keys, and are virtually limitless.
>>
>>
Ideally, I'd like to send some sort of purge event on keys a couple of
>>
days later, where I empty the aggregate in the fold. That still leaves
>>
the key though, where does that go?
>>
>>
Any answers highly appreciated...
>>
>>
Greetings
>>
>>
Bart
