They would be awesome, but it’s not yet possible in Flink Streaming, I’m afraid.
> On 18 Feb 2016, at 10:59, Stefano Baghino <stefano.bagh...@radicalbit.io> > wrote: > > I think combiners are pretty awesome for certain cases to minimize network > usage (the average use case seems to fit perfectly), maybe it would be > worthwhile adding a detailed description of the approach to the docs? > > On Thu, Feb 18, 2016 at 10:47 AM, Aljoscha Krettek <aljos...@apache.org> > wrote: > @Nirmalya: Yes, this is right if you temperatures don’t have any other field > on which you could partition them. > > @Stefano: Under some circumstances it would be possible to use a a combiner > (I’m using the name as Hadoop MapReduce would use it, here). When the > assignment of elements to windows happens based on the timestamp in the > elements and window triggering happens based on watermark it is possible to > combine locally. The reason is that the elements will end up in the same > windows regardless of the time at which the window is processed so it can be > done in two steps. Does that make sense? It’s a very ad-hoc description and I > could make up a drawing or something if that helped. :D > > > > On 18 Feb 2016, at 10:04, Stefano Baghino <stefano.bagh...@radicalbit.io> > > wrote: > > > > Thanks, Aljosha, for the explanation. Isn't there a way to apply the > > concept of the combiner to a streaming process? > > > > On Thu, Feb 18, 2016 at 3:56 AM, Nirmalya Sengupta > > <sengupta.nirma...@gmail.com> wrote: > > Hello Aljoscha <aljos...@apache.org> > > > > Thanks very much for clarifying the role of Pre-Aggregation (rather, > > Incr-Aggregation, now that I understand the intention). It helps me to > > understand. Thanks to Setfano too, for keeping at the original question of > > mine. > > > > My current understanding is that if I have to compute the average of a > > streaming set of _temperatures_ then the *best* way to accomplish this, is > > by employing one node (or thread, on my laptop), losing speed but gaining > > deterministic behaviour in the process. I can decide to capture the average > > either by grouping the temperatures by count or by time. Because I am > > sliding the window anyway, I don't run the risk of accumulation of elements > > in the window and buffer overrun. > > > > Could you please confirm if my understanding is correct? I feel happy if I > > 'understand' the basis of a design well! :-) > > > > -- Nirmalya > > -- > > Software Technologist > > http://www.linkedin.com/in/nirmalyasengupta > > "If you have built castles in the air, your work need not be lost. That is > > where they should be. > > Now put the foundation under them." > > > > > > > > -- > > BR, > > Stefano Baghino > > > > Software Engineer @ Radicalbit > > > > > -- > BR, > Stefano Baghino > > Software Engineer @ Radicalbit
