Colin Shaw created KAFKA-14739:
----------------------------------
Summary: Kafka consumer reading messages out of order after a
rebalance
Key: KAFKA-14739
URL: https://issues.apache.org/jira/browse/KAFKA-14739
Project: Kafka
Issue Type: Bug
Affects Versions: 2.6.0
Reporter: Colin Shaw
Attachments: image-2023-02-21-16-45-10-464.png
We are seeing often when our consumers are running and a rebalance is
triggered, some partitions do not resume at the correct offset causing a large
number of events to be skipped.
Our understanding is that within a partition events should also be read in
chronological order.
These consumers run once a day and we see that when they resume again the next
day it does return to the missed events.
An example we investigated:
{quote}We see for partition 88 starting processing around Offset 14001. Events
were processed until Offset 14059 but we could see Offset 14060 was not
processed from out logs.
Upon investigation we could see a rebalance had occured after offset 14059 and
afterwards the next event processed was offset 14190 causing about 130 events
to be skipped on this partition.
The subsequent day we it did resume at Offset 14060
{quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
