A. Sophie Blee-Goldman created KAFKA-12475:
----------------------------------------------
Summary: Kafka Streams breaks EOS with remote state stores
Key: KAFKA-12475
URL: https://issues.apache.org/jira/browse/KAFKA-12475
Project: Kafka
Issue Type: Bug
Components: streams
Reporter: A. Sophie Blee-Goldman
Currently in Kafka Streams, exactly-once semantics (EOS) require that the state
stores be completely erased and restored from the changelog from scratch in
case of an error. This erasure is implemented by closing the state store and
then simply wiping out the local state directory. This works fine for the two
store implementations provided OOTB, in-memory and rocksdb, but fails when the
application includes a custom StateStore based on remote storage, such as
Redis. In this case Streams will fail to erase any of the data before
reinserting data from the changelog, resulting in possible duplicates and
breaking the guarantee of EOS.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
