[
https://issues.apache.org/jira/browse/KAFKA-9724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17069028#comment-17069028
]
ASF GitHub Bot commented on KAFKA-9724:
---------------------------------------
mumrah commented on pull request #8376: KAFKA-9724 Newer clients not always
sending fetch request to older brokers
URL: https://github.com/apache/kafka/pull/8376
We had a similar case previously with KAFKA-8422 (#6806) where we would skip
the validation step if the broker was on a version older than 2.3.
This PR makes a similar change on the `prepareFetchRequest` side. If the
broker is older than 2.3, we will skip the transition to AWAITING_VALIDATION
but also we will clear that state if it had been set by a call to
`seekUnvalidated`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Consumer wrongly ignores fetched records "since it no longer has valid
> position"
> --------------------------------------------------------------------------------
>
> Key: KAFKA-9724
> URL: https://issues.apache.org/jira/browse/KAFKA-9724
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Affects Versions: 2.4.0
> Reporter: Oleg Muravskiy
> Priority: Major
>
> After upgrading kafka-client to 2.4.0 (while brokers are still at 2.2.0)
> consumers in a consumer group intermittently stop progressing on assigned
> partitions, even when there are messages to consume. This is not a permanent
> condition, as they progress from time to time, but become slower with time,
> and catch up after restart.
> Here is a sample of 3 consecutive ignored fetches:
> {noformat}
> 2020-03-15 12:08:58,440 DEBUG [Thread-6] o.a.k.c.c.i.ConsumerCoordinator -
> Committed offset 538065584 for partition mrt-rrc10-6
> 2020-03-15 12:08:58,541 DEBUG [Thread-6] o.a.k.c.consumer.internals.Fetcher -
> Skipping validation of fetch offsets for partitions [mrt-rrc10-1,
> mrt-rrc10-6, mrt-rrc22-7] since the broker does not support the required
> protocol version (introduced in Kafka 2.3)
> 2020-03-15 12:08:58,549 DEBUG [Thread-6] org.apache.kafka.clients.Metadata -
> Updating last seen epoch from null to 62 for partition mrt-rrc10-6
> 2020-03-15 12:08:58,557 DEBUG [Thread-6] o.a.k.c.c.i.ConsumerCoordinator -
> Committed offset 538065584 for partition mrt-rrc10-6
> 2020-03-15 12:08:58,652 DEBUG [Thread-6] o.a.k.c.consumer.internals.Fetcher -
> Skipping validation of fetch offsets for partitions [mrt-rrc10-1,
> mrt-rrc10-6, mrt-rrc22-7] since the broker does not support the required
> protocol version (introduced in Kafka 2.3)
> 2020-03-15 12:08:58,659 DEBUG [Thread-6] org.apache.kafka.clients.Metadata -
> Updating last seen epoch from null to 62 for partition mrt-rrc10-6
> 2020-03-15 12:08:58,659 DEBUG [Thread-6] o.a.k.c.consumer.internals.Fetcher -
> Fetch READ_UNCOMMITTED at offset 538065584 for partition mrt-rrc10-6 returned
> fetch data (error=NONE, highWaterMark=538065631, lastStableOffset =
> 538065631, logStartOffset = 485284547, preferredReadReplica = absent,
> abortedTransactions = null, recordsSizeInBytes=16380)
> 2020-03-15 12:08:58,659 DEBUG [Thread-6] o.a.k.c.consumer.internals.Fetcher -
> Ignoring fetched records for partition mrt-rrc10-6 since it no longer has
> valid position
> 2020-03-15 12:08:58,665 DEBUG [Thread-6] o.a.k.c.c.i.ConsumerCoordinator -
> Committed offset 538065584 for partition mrt-rrc10-6
> 2020-03-15 12:08:58,761 DEBUG [Thread-6] o.a.k.c.consumer.internals.Fetcher -
> Skipping validation of fetch offsets for partitions [mrt-rrc10-1,
> mrt-rrc10-6, mrt-rrc22-7] since the broker does not support the required
> protocol version (introduced in Kafka 2.3)
> 2020-03-15 12:08:58,761 DEBUG [Thread-6] o.a.k.c.consumer.internals.Fetcher -
> Added READ_UNCOMMITTED fetch request for partition mrt-rrc10-6 at position
> FetchPosition{offset=538065584, offsetEpoch=Optional[62],
> currentLeader=LeaderAndEpoch{leader=node03.kafka:9092 (id: 3 rack: null),
> epoch=-1}} to node node03.kafka:9092 (id: 3 rack: null)
> 2020-03-15 12:08:58,761 DEBUG [Thread-6] o.a.k.c.consumer.internals.Fetcher -
> Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(),
> implied=(mrt-rrc10-6, mrt-rrc22-7, mrt-rrc10-1)) to broker node03.kafka:9092
> (id: 3 rack: null)
> 2020-03-15 12:08:58,770 DEBUG [Thread-6] org.apache.kafka.clients.Metadata -
> Updating last seen epoch from null to 62 for partition mrt-rrc10-6
> 2020-03-15 12:08:58,770 DEBUG [Thread-6] o.a.k.c.consumer.internals.Fetcher -
> Fetch READ_UNCOMMITTED at offset 538065584 for partition mrt-rrc10-6 returned
> fetch data (error=NONE, highWaterMark=538065727, lastStableOffset =
> 538065727, logStartOffset = 485284547, preferredReadReplica = absent,
> abortedTransactions = null, recordsSizeInBytes=51864)
> 2020-03-15 12:08:58,770 DEBUG [Thread-6] o.a.k.c.consumer.internals.Fetcher -
> Ignoring fetched records for partition mrt-rrc10-6 since it no longer has
> valid position
> 2020-03-15 12:08:58,808 DEBUG [Thread-6] o.a.k.c.c.i.ConsumerCoordinator -
> Committed offset 538065584 for partition mrt-rrc10-6
> {noformat}
> After which consumer makes progress:
> {noformat}
> 2020-03-15 12:08:58,871 DEBUG [Thread-6] o.a.k.c.consumer.internals.Fetcher -
> Skipping validation of fetch offsets for partitions [mrt-rrc10-1,
> mrt-rrc10-6, mrt-rrc22-7] since the broker does not support the required
> protocol version (introduced in Kafka 2.3)
> 2020-03-15 12:08:58,871 DEBUG [Thread-6] o.a.k.c.consumer.internals.Fetcher -
> Added READ_UNCOMMITTED fetch request for partition mrt-rrc10-6 at position
> FetchPosition{offset=538065584, offsetEpoch=Optional[62],
> currentLeader=LeaderAndEpoch{leader=node03.kafka:9092 (id: 3 rack: null),
> epoch=-1}} to node node03.kafka:9092 (id: 3 rack: null)
> 2020-03-15 12:08:58,871 DEBUG [Thread-6] o.a.k.c.consumer.internals.Fetcher -
> Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(),
> implied=(mrt-rrc10-6, mrt-rrc22-7, mrt-rrc10-1)) to broker node03.kafka:9092
> (id: 3 rack: null)
> 2020-03-15 12:08:58,872 DEBUG [Thread-6] o.a.k.c.consumer.internals.Fetcher -
> Fetch READ_UNCOMMITTED at offset 538065584 for partition mrt-rrc10-6 returned
> fetch data (error=NONE, highWaterMark=538065744, lastStableOffset =
> 538065744, logStartOffset = 485284547, preferredReadReplica = absent,
> abortedTransactions = null, recordsSizeInBytes=58293)
> 2020-03-15 12:08:58,872 DEBUG [Thread-6] o.a.k.c.consumer.internals.Fetcher -
> Added READ_UNCOMMITTED fetch request for partition mrt-rrc10-6 at position
> FetchPosition{offset=538065744, offsetEpoch=Optional[62],
> currentLeader=LeaderAndEpoch{leader=node03.kafka:9092 (id: 3 rack: null),
> epoch=-1}} to node node03.kafka:9092 (id: 3 rack: null)
> 2020-03-15 12:08:58,872 DEBUG [Thread-6] o.a.k.c.consumer.internals.Fetcher -
> Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(mrt-rrc10-6),
> toForget=(), implied=(mrt-rrc22-7, mrt-rrc10-1)) to broker node03.kafka:9092
> (id: 3 rack: null)
> 2020-03-15 12:08:58,880 DEBUG [Thread-6] org.apache.kafka.clients.Metadata -
> Updating last seen epoch from null to 62 for partition mrt-rrc10-6
> 2020-03-15 12:08:58,885 DEBUG [Thread-6] o.a.k.c.c.i.ConsumerCoordinator -
> Committed offset 538065744 for partition mrt-rrc10-6
> {noformat}
> But it could be stuck for quite a long time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
