[ https://issues.apache.org/jira/browse/KAFKA-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599002#comment-17599002 ]
John Gray edited comment on KAFKA-14172 at 9/1/22 2:44 PM: ----------------------------------------------------------- I know next to nothing about the internal workings of Kafka, sadly, but I am noticing that KAFKA-12486 was introduced in 3.1.0, which is the version I started noticing problems. I notice you helped out with that Jira, [~ableegoldman] , is there any possible way in your mind that it might cause weirdness with state restoration? was (Author: gray.john): I know next to nothing about the internal workings of Kafka, sadly, but I am noticing that KAFKA-12486 was introduced in 3.1.0, which is the version I started noticing problems. I notice you helped out with that Jira, [~ableegoldman] , is there any possible way in your mind it might cause weirdness with state restoration? > bug: State stores lose state when tasks are reassigned under EOS wit… > --------------------------------------------------------------------- > > Key: KAFKA-14172 > URL: https://issues.apache.org/jira/browse/KAFKA-14172 > Project: Kafka > Issue Type: Bug > Components: streams > Affects Versions: 3.1.1 > Reporter: Martin Hørslev > Priority: Major > > h1. State stores lose state when tasks are reassigned under EOS with standby > replicas and default acceptable lag. > I have observed that state stores used in a transform step under a Exactly > Once semantics ends up losing state after a rebalancing event that includes > reassignment of tasks to previous standby task within the acceptable standby > lag. > > The problem is reproduceable and an integration test have been created to > showcase the [issue|https://github.com/apache/kafka/pull/12540]. > A detailed description of the observed issue is provided > [here|https://github.com/apache/kafka/pull/12540/files?short_path=3ca480e#diff-3ca480ef093a1faa18912e1ebc679be492b341147b96d7a85bda59911228ef45] > Similar issues have been observed and reported to StackOverflow for example > [here|https://stackoverflow.com/questions/69038181/kafka-streams-aggregation-data-loss-between-instance-restarts-and-rebalances]. > -- This message was sent by Atlassian Jira (v8.20.10#820010)