Hi Manu, Yes. If a StreamTask recovers, it will write to the same change log's topic partition. Log compaction is enable per default for those topics. You still might see some duplicates in your output. Currently, Kafka Streams guarantees at-least-once processing (exactly-once processing is on the roadmap though).
-Matthias On 05/23/2016 06:10 AM, Manu Zhang wrote: > Hi All, > > I'm new to Kafka Streams and have a question on change log. > If a StreamTask fails and is restarted, will the change log be written to > the old change log's topic partition ? Is it possible for some change log > topic partition to have duplicate records so that log compaction is > required ? > > Thanks, > Manu Zhang >
signature.asc
Description: OpenPGP digital signature
