[
https://issues.apache.org/jira/browse/KAFKA-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15709345#comment-15709345
]
Dong Lin commented on KAFKA-4415:
---------------------------------
[~ijuma] As of current Kafka implementation, Kafka does send O(partitionNum *
brokerNum) MetadataUpdateRequest when responding to ControlledShutdownRequest,
where each MetadataUpdateRequest will contain only one partition. The
optimization in https://github.com/apache/kafka/pull/2141 can reduce it to
O(brokerNum) MetadataUpdateRequest, where each request includes O(partitionNum)
partitions. Originally I have included that fix in KAFKA-4415 as well. But I
moved it out of KAFKA-4415 and put into KAFKA-4444 because there is un-resolved
problem in that optimization.
I just updated the description of JIRA to make it accurate.
> Reduce time to create and send MetadataUpdateRequest
> ----------------------------------------------------
>
> Key: KAFKA-4415
> URL: https://issues.apache.org/jira/browse/KAFKA-4415
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.10.1.0
> Reporter: Dong Lin
> Assignee: Dong Lin
> Fix For: 0.10.2.0
>
>
> As of current implementation, when controller receives
> ControlledShutdownRequest, it will 1) for every broker in the cluster, for
> every partition on the broker which wants to shutdown, create an instance of
> PartitionStateInfo and add it to
> ControllerBrokerRequestBatch.,updateMetadataRequestMap; and 2) for every
> broker, for every follower partitions on the broker which wants to shutdown,
> send one MetadataUpdateRequst to that broker.
> In order to shutdown a broker, the controller will need to instantiate
> O(partitionNum * brokerNum) PartitionStateInfo. This is not efficient. The
> broker should only need to instantiate O(partitionNum) PartitionStateInfo.
> Micro-benchmark results show that this optimization can reduce the time of
> processing ControlledShutdownRequest by 30%.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
