RE: KSTREAM-AGGREGATE-STATE-STORE persistence?

2019-08-12 Thread Tim Ward
Thanks! Tim Ward -Original Message- From: Bruno Cadonna Sent: 12 August 2019 14:18 To: users@kafka.apache.org Subject: Re: KSTREAM-AGGREGATE-STATE-STORE persistence? Hi Tim, Kafka Streams guarantees at-least-once processing semantics by default. That means, a record is processed (e.g

Re: KSTREAM-AGGREGATE-STATE-STORE persistence?

2019-08-12 Thread Bruno Cadonna
Hi Tim, Kafka Streams guarantees at-least-once processing semantics by default. That means, a record is processed (e.g. added to an aggregate) at least once but might be processed multiple times. The cause for processing the same record multiple time are crashes as you described. Exactly-once proc

KSTREAM-AGGREGATE-STATE-STORE persistence?

2019-08-12 Thread Tim Ward
I believe I have witnessed - at least twice - something like the following happening, in a Kafka Streams application where I have a .groupByKey().windowedBy().aggregate() sequence. * Application runs for a while * Application crashes * Application restarts * Aggregator.apply() i