Re: How to trigger only once for each window when using TumblingProcessingTimeWindows?

2020-08-19 Thread David Anderson
The purpose of the reduce() and aggregate() methods on windows is to allow
for incremental computation of window results. This has two principal
advantages: (1) the computation of the results is spread out, rather than
occurring all in one go at the end of each window, thereby reducing the
likelihood of spiky loads, and (2) this eliminates the need to buffer the
window contents.

Despite the result being computed incrementally, by default it will only be
emitted once, at the end of the window, when the window is triggered.

If you prefer to have the window's contents gathered into an Iterable which
you can process at the end of the window, then use a ProcessWindowFunction
[1].

By the way, in the Apache Flink training section of the docs there is a
tutorial covering the Window API [2] that includes examples written in both
of these styles [3].

[1]
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/windows.html#processwindowfunction
[2]
https://ci.apache.org/projects/flink/flink-docs-stable/learn-flink/streaming_analytics.html#windows
[3]
https://ci.apache.org/projects/flink/flink-docs-stable/learn-flink/streaming_analytics.html#window-functions

Regards,
David

On Wed, Aug 19, 2020 at 12:11 PM wangl...@geekplus.com <
wangl...@geekplus.com> wrote:

>
> Read Kafka message and keyBy by tableName, then write the message list to
> DataBase with batchUpdate
>
> keyedStream.window(TumblingProcessingTimeWindows.of(Time.seconds(1))).aggregate(new
>  ListAggregate()).addSink(new TemplateMySQLSink());
>
>
> It seems that for every record comming, the aggregate function will be 
> trigged.
>
> But I want to trigger only once for every window.
>
> How can i implement this?
>
>
> Thanks,
>
> Lei
>
>
> --
> wangl...@geekplus.com 
>
>


How to trigger only once for each window when using TumblingProcessingTimeWindows?

2020-08-19 Thread wangl...@geekplus.com

Read Kafka message and keyBy by tableName, then write the message list to 
DataBase with batchUpdate
keyedStream.window(TumblingProcessingTimeWindows.of(Time.seconds(1))).aggregate(new
 ListAggregate()).addSink(new TemplateMySQLSink());

It seems that for every record comming, the aggregate function will be trigged. 
But I want to trigger only once for every window.How can i implement this?
Thanks,Lei


wangl...@geekplus.com