Re: Kafka Streams windowed aggregation

2016-10-05 Thread Eno Thereska
Hi Davood, The behaviour is indeed as you say. Recently we checked in KIP-63 in trunk (it will be part of the 0.10.1 release coming up). That should reduce the amount of downstream traffic you see (https://cwiki.apache.org/confluence/display/KAFKA/KIP-63%3A+Unify+store+and+downstream+caching+in

Kafka Streams windowed aggregation

2016-10-05 Thread Davood Rafiei
Hi, I want to do windowed aggregation with streams library. However, I get the output from particular operator immediately, independent of window size. This makes sense for unlimited windows or sometimes for event time windows. However, for ingestion time or processing time windows, users may want