[
https://issues.apache.org/jira/browse/KAFKA-15079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Jacot reassigned KAFKA-15079:
-----------------------------------
Assignee: Ritika Muduganti
> Offset expirations in topics with less event generation results in messages
> that take months to be delivered.
> -------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-15079
> URL: https://issues.apache.org/jira/browse/KAFKA-15079
> Project: Kafka
> Issue Type: Bug
> Components: group-coordinator
> Reporter: Ritika Muduganti
> Assignee: Ritika Muduganti
> Priority: Major
>
> It was observed that messages were taking months to be delivered in certain
> pipelines. The offsets had been reset and consumption restarted from messages
> that were a couple months old.
> On further investigation the team found that once the
> offsets.retention.minutes (7 days) elapses and there's no subscribers to the
> topic, the offsets are subject to immediate expiration. This can be a problem
> when we close the current consumer and the retention period has already
> passed due to the infrequent nature of offset commits. The offsets then
> expire before a new consumer can come up and subscribe to the topic again.
> Since only one consumer could be subscribed to the topic it will be removed
> from the subscription list as soon as the consumer leaves.
> Two things to keep in mind here are :-
> # The consumer group has different consumers subscribed to different topics.
> # Events aren't generated very frequently to these topics.
> [https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/coordinator/group/GroupMetadata.scala#LL766C1-L825C4]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
