[
https://issues.apache.org/jira/browse/KAFKA-19781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18029325#comment-18029325
]
Arpit Goyal commented on KAFKA-19781:
-------------------------------------
This is what i understand , can can try reproducing it through test also
. 1. Partitions marked for revocation (pendingRevocation = true) are excluded
from fetching but NOT from position updates
2. shouldInitialize() only checks if state is INITIALIZING, ignoring
pendingRevocation
3. hasValidPosition() only checks fetch state, ignoring pendingRevocation
4. This causes updateFetchPositions() to attempt fetching offsets for
partitions being revoked
> Consumer NoOffsetForPartitionException for partitions being revoked
> -------------------------------------------------------------------
>
> Key: KAFKA-19781
> URL: https://issues.apache.org/jira/browse/KAFKA-19781
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Reporter: Lianet Magrans
> Assignee: Lianet Magrans
> Priority: Major
> Fix For: 4.2.0
>
>
> Currently, the consumer attempts to update positions for all partitions it
> owns that don't have a position, even when the partition may be already being
> revoked. This could lead to NoOffsetForPartitionException for such partitions
> (if there is no reset strategy defined). There are 2 main issues around this:
> * is probably unneeded work to fetch offsets and update positions for
> partitions being revoked. At that point we're actually not allowing fetching
> from there anymore
> * throwing NoOffsetForPartitionException may lead applications to take
> action to set positions themselves, which will most probably fail (the
> partition will most probably not owned by the consumer anymore since it was
> already being revoked when the NoOffsetForPartitionException was generated).
> We've seen this on Kafka Streams, that handled NoOffsetForPartitionException
> by calling seek to set positions.
> This task is to review if there are no other implications I may be missing?
> Then fix to ensure we don't update positions for partitions being revoked
> (aligned with how we don't allow fetching from them)
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
