Streaming cannot use windows. The aggregations happen on the trigger. 

The elements being aggregated are only known after the trigger delivers the 
elements to the evaluation function. 

Since windows can overlap and even assignment to a window is not done until the 
elements arrive at the sum operator in your case, combiner cannot know what to 
pre aggregate even if were available. 



> On Aug 11, 2016, at 8:51 PM, Elias Levy <fearsome.lucid...@gmail.com> wrote:
> 
> I am wondering if Flink makes use of combiners to pre-reduce a keyed and 
> windowed stream before shuffling the data among workers.
> 
> I.e. will it use a combiner in something like:
> 
> stream.flatMap {...}
>       .assignTimestampsAndWatermarks(...)
>       .keyBy(...)
>       .timeWindow(...)
>       .trigger(...)
>       .sum("cnt")
> 
> or will it shuffle the keyed input before the sum reduction?
> 
> If it does make use of combiners, it would be useful to point this out in the 
> documentation, particularly if it only applies to certain types of reducers, 
> folds, etc.

Reply via email to