John Roesler created KAFKA-8204:
-----------------------------------
Summary: Streams may flush state stores in the incorrect order
Key: KAFKA-8204
URL: https://issues.apache.org/jira/browse/KAFKA-8204
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: John Roesler
Assignee: John Roesler
Cached state stores may forward records during a flush call, so Streams should
flush the stores in topological order. Otherwise, Streams may flush a
downstream store before an upstream one, resulting in sink results being
committed without the corresponding state changelog updates being committed.
This behavior is partly responsible for the bug reported in KAFKA-7895 .
The fix is simply to flush the stores in topological order, then when the
upstream store forwards records to a downstream stateful processor, the
corresponding state changes will be correctly flushed as well.
An alternative would be to repeatedly call flush on all state stores until they
report there is nothing left to flush, but this requires a public API change to
enable state stores to report whether they need a flush or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
