[
https://issues.apache.org/jira/browse/KAFKA-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599478#comment-17599478
]
John Gray commented on KAFKA-14172:
-----------------------------------
[~Horslev] we utilize static membership for our stateful apps, so for us the
cluster upgrades seem to be about the only time they really rebalance. So I do
think we lose data because of the rebalancing EOS consumers. The biggest
difference I think is that we don't use standby replicas, yet still are getting
data loss during restores.
> bug: State stores lose state when tasks are reassigned under EOS wit…
> ---------------------------------------------------------------------
>
> Key: KAFKA-14172
> URL: https://issues.apache.org/jira/browse/KAFKA-14172
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 3.1.1
> Reporter: Martin Hørslev
> Priority: Major
>
> h1. State stores lose state when tasks are reassigned under EOS with standby
> replicas and default acceptable lag.
> I have observed that state stores used in a transform step under a Exactly
> Once semantics ends up losing state after a rebalancing event that includes
> reassignment of tasks to previous standby task within the acceptable standby
> lag.
>
> The problem is reproduceable and an integration test have been created to
> showcase the [issue|https://github.com/apache/kafka/pull/12540].
> A detailed description of the observed issue is provided
> [here|https://github.com/apache/kafka/pull/12540/files?short_path=3ca480e#diff-3ca480ef093a1faa18912e1ebc679be492b341147b96d7a85bda59911228ef45]
> Similar issues have been observed and reported to StackOverflow for example
> [here|https://stackoverflow.com/questions/69038181/kafka-streams-aggregation-data-loss-between-instance-restarts-and-rebalances].
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
