sanghyeok An created KAFKA-21430:
------------------------------------
Summary: Include metadata offset in Metadata responses to help
clients detect stale metadata
Key: KAFKA-21430
URL: https://issues.apache.org/jira/browse/KAFKA-21430
Project: Kafka
Issue Type: Improvement
Components: clients, core
Reporter: sanghyeok An
Assignee: sanghyeok An
*Background*
In KRaft mode, brokers may have applied the metadata log up to different
offsets. As a result, a client may receive a Metadata response representing an
older state than the information it has already cached.
Clients currently compare per-partition {{leaderEpoch}} values to reject some
stale information. However, this does not detect stale metadata for changes
such as ISR updates within the same leader epoch or changes to the broker list.
*Proposal*
Include the offset of the metadata image used to generate a Metadata response,
allowing clients to compare it with the offset associated with their cached
information.
Within the same cluster, clients should retain cached information when incoming
information is based on a lower offset. For requests covering only a subset of
topics, comparisons should use the offset associated with the corresponding
cached information.
*Expected Benefits*
Prevent metadata propagation delays between brokers from causing clients to
overwrite their caches with older information, and support freshness
comparisons for metadata changes that cannot be detected using {{leaderEpoch}}
alone.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
