Dimitar Dimitrov created KAFKA-18583:
----------------------------------------
Summary: Improve/fix
KRaftMetadataCache.getPartitionReplicaEndpoints
Key: KAFKA-18583
URL: https://issues.apache.org/jira/browse/KAFKA-18583
Project: Kafka
Issue Type: Task
Components: core
Affects Versions: 3.8.1, 3.7.2, 3.9.0
Reporter: Dimitar Dimitrov
Assignee: Dimitar Dimitrov
The {{KRaftMetadataCache.getPartitionReplicaEndpoints}} method, called when a
replica selector is being used (e.g. for Fetch From Follower), currently takes
a single {{TopicPartition}} for which to calculate node endpoints per replica
ID, but then goes on to process all the partitions of the topic. As a result it
seems to both calculate endpoints which are not relevant for the given
partition and to recalculate the relevant endpoints multiple times as replicas
of many different partitions.
This could be very detrimental to performance if it touches topics with larger
number of partitions.
Unlike its KRaft counterpart, {{ZkMetadataCache.getPartitionReplicaEndpoints}}
seems to do the right thing, processing only the partition info of the given
partition. Also the problem doesn't seem to be a recent regression - it looks
like it was present in the original form of {{KRaftMetadataCache}} introduced
more than 3 years ago.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
