With pre-aggregation (which the Reduce does), Flink can handle many windows
and many keys, as long as you have the memory and storage to support that.
Your case should work.
On Mon, Feb 20, 2017 at 4:58 PM, Vadim Vararu
wrote:
> It's something like:
>
>
It's something like:
DataStreamSource stream =
env.addSource(getKafkaConsumer(parameterTool)); stream
.map(getEventToDomainMapper())
.keyBy(getKeySelector())
.window(ProcessingTimeSessionWindows.withGap(Time.minutes(90)))
.reduce(getReducer())
Hi Vadim,
this of course depends on your use case. The question is how large is
your state per pane and how much memory is available for Flink?
Are you using incremental aggregates such that only the aggregated value
per pane has to be kept in memory?
Regards,
Timo
Am 20/02/17 um 16:34
HI guys,
Is it okay to have very many (tens of thousands or hundreds of thousand)
of session windows?
Thanks, Vadim.