Jeff Kim created KAFKA-14075:
--------------------------------
Summary: Consumer Group deletion does not delete pending
transactional offset commits
Key: KAFKA-14075
URL: https://issues.apache.org/jira/browse/KAFKA-14075
Project: Kafka
Issue Type: Bug
Reporter: Jeff Kim
Assignee: Jeff Kim
In
[GroupMetadata.removeAllOffsets()|https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/coordinator/group/GroupMetadata.scala#L729-L740]
we pass in the offsets cache to delete pendingTransactionalOffsetCommits upon
group deletion. So only transactional offset commits for topic partitions
already in the offsets cache will be deleted.
However, we add a transactional offset commit to the offsets cache only after
the commit/abort marker is written to the log in
[GroupMetadata.completePendingTxnOffsetCommit()|https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/coordinator/group/GroupMetadata.scala#L692]
So even after a group deletion we can still have pending transactional offset
commits for a group that's supposed to be deleted. The group metadata manager
will throw an IllegalStateException
[here|https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala#L740]
while loading group to memory. We will hit this exception on every load to
group as long as the hanging transaction never completes.
We should delete all pending transactional offset commits (instead of only
topic partitions that exist in the offsets cache) when a group is deleted in
GroupMetadata.removeOffsets()
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
