[
https://issues.apache.org/jira/browse/KAFKA-20040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lianet Magrans updated KAFKA-20040:
-----------------------------------
Description:
When using topic IDs for fetching committed offsets (as of 4.2), there could be
a race where a consumer requests offsets to update positions, but unsubscribes
from it before receiving a response. In that case, the response to the
OffsetFetch will contain a topic ID that the consumer cannot resolve from
metadata. Currently the OffsetFetch response handling fails with:
{code:java}
Error fetching committed offsets to update positions
(org.apache.kafka.clients.consumer.internals.OffsetsRequestManager:424)
org.apache.kafka.common.KafkaException: Topic does not exist
{code}
Note that this would never be the case of manual calls to fetch offsets
(consumer.committed). That API directly adds the topics to metadata as
transient topics, and keeps them until a response is received.
was:
When using topic IDs for fetching committed offsets (as of 4.2), there could be
a race where a consumer requests offsets to update positions, but unsubscribes
from it before receiving a response. In that case, the response to the
OffsetFetch will contain a topic ID that the consumer cannot resolve from
metadata. Currently the OffsetFetch response handling fails with:
{code:java}
Error fetching committed offsets to update positions
(org.apache.kafka.clients.consumer.internals.OffsetsRequestManager:424)
org.apache.kafka.common.KafkaException: Topic does not exist
{code}
We should consider ignoring the topic that the consumer no longer needs
* it does exist on the broker
* the consumer no longer needs offsets to update positions (it unsubscribed
from the topic)
* note that this would never be the case of manual calls to fetch offsets
(consumer.committed). That API directly adds the topics to metadata as
transient topics, and keeps them until a response is received.
> Race condition on consumer updating positions and unsubscribe when using
> committed offsets with topic ID
> --------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-20040
> URL: https://issues.apache.org/jira/browse/KAFKA-20040
> Project: Kafka
> Issue Type: Bug
> Reporter: Lianet Magrans
> Assignee: Lianet Magrans
> Priority: Major
>
> When using topic IDs for fetching committed offsets (as of 4.2), there could
> be a race where a consumer requests offsets to update positions, but
> unsubscribes from it before receiving a response. In that case, the response
> to the OffsetFetch will contain a topic ID that the consumer cannot resolve
> from metadata. Currently the OffsetFetch response handling fails with:
>
> {code:java}
> Error fetching committed offsets to update positions
> (org.apache.kafka.clients.consumer.internals.OffsetsRequestManager:424)
> org.apache.kafka.common.KafkaException: Topic does not exist
> {code}
>
> Note that this would never be the case of manual calls to fetch offsets
> (consumer.committed). That API directly adds the topics to metadata as
> transient topics, and keeps them until a response is received.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
