Rafał Boniecki created KAFKA-9543:
-------------------------------------
Summary: Consumer offset reset after new segment rolling
Key: KAFKA-9543
URL: https://issues.apache.org/jira/browse/KAFKA-9543
Project: Kafka
Issue Type: Bug
Affects Versions: 2.4.0
Reporter: Rafał Boniecki
Attachments: Untitled.png
After upgrade from kafka 2.1.1 to 2.4.0, I'm experiencing unexpected consumer
offset resets.
Consumer:
{code:java}
2020-02-12T11:12:58.402+01:00 hostname 4a2a39a35a02
[2020-02-12T11:12:58,402][INFO
][org.apache.kafka.clients.consumer.internals.Fetcher] [Consumer
clientId=logstash-1, groupId=logstash] Fetch offset 1632750575 is out of range
for partition stats-5, resetting offset
{code}
Broker:
{code:java}
2020-02-12 11:12:58:400 CET INFO
[data-plane-kafka-request-handler-1][kafka.log.Log] [Log partition=stats-5,
dir=/kafka4/data] Rolled new log segment at offset 1632750565 in 2 ms.{code}
All resets are perfectly correlated to rolling new segments at the broker -
segment is rolled first, then, couple of ms later, reset on the consumer
occurs. Attached is grafana graph with consumer lag per partition. All sudden
spikes in lag are offset resets due to this bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
