rajinisivaram commented on a change in pull request #9622:
URL: https://github.com/apache/kafka/pull/9622#discussion_r544180651
##########
File path: core/src/main/scala/kafka/server/MetadataCache.scala
##########
@@ -314,9 +315,16 @@ class MetadataCache(brokerId: Int) extends Logging {
error(s"Listeners are not identical across brokers: $aliveNodes")
}
+ val newTopicIds = updateMetadataRequest.topicStates().asScala
+ .map(topicState => (topicState.topicName(), topicState.topicId()))
+ .filter(_._2 != Uuid.ZERO_UUID).toMap
+ val topicIds = mutable.Map.empty[String, Uuid]
+ topicIds.addAll(metadataSnapshot.topicIds)
+ topicIds.addAll(newTopicIds)
Review comment:
When a topic is deleted, brokers process UpdateMetadataRequest and
remove deleted topics from their cache. We track deletion state in ZooKeeper
and as you mentioned, you can get this information by directly going to ZK in
kafka-topics.sh. But we don't retain that information in every broker. I would
remove topic id in the code segment just below this when the partition state is
removed from the MetadataCache since we cannot clearly have a map that keeps
growing in brokers. Is there a reason why we would want to retain topic id in
every broker even after the topic has been deleted? We can't get this
information through existing metadata request from brokers anyway. I guess in
future, we can add additional metadata to track deleted topic ids if we wanted
to, but for now it seems better to delete topic ids from MetadataCache when we
delete the partition from the cache. What do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
