Vinoth Chandar created KAFKA-8870:
-------------------------------------
Summary: Prevent dirty reads of Streams state store from
Interactive queries
Key: KAFKA-8870
URL: https://issues.apache.org/jira/browse/KAFKA-8870
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: Vinoth Chandar
Today, Interactive Queries (IQ) against Streams state store could see
uncommitted data, even with EOS processing guarantees (these are actually
orthogonal, but clarifying since EOS may give the impression that everything is
dandy). This is causes primarily because state updates in rocksdb are visible
even before the kafka transaction is committed. Thus, if the instance fails,
then the failed over instance will redo the uncommited old transaction and the
following could be possible during recovery,.
Value for key K can go from *V0 → V1 → V2* on active instance A, IQ reads V1,
instance A fails and any failure/rebalancing will leave the standy instance B
rewinding offsets and reprocessing, during which time IQ can again see V0 or V1
or any number of previous values for the same key.
In this issue, we will plan work towards providing consistency for IQ, for a
single row in a single state store. i.e once a query sees V1, it can only see
either V1 or V2.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
