[
https://issues.apache.org/jira/browse/KAFKA-8180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17130763#comment-17130763
]
David Jacot commented on KAFKA-8180:
------------------------------------
I have improved the batching to be more aggressive when a batch of topics is
deleted back in Feb: https://issues.apache.org/jira/browse/KAFKA-9499. This
reduces the processing time of the topic deletion event, reduces the number of
requests sent out and therefore reduces the amount of work for the controller.
I think that the pain point mentioned in this ticket remains valid but the bar
should be higher.
[~wushujames] We are working on KIP-599 to mitigate this in the long run. Check
it out.
> Deleting large number of topics can block the controller for the time it
> takes to delete all of them
> ----------------------------------------------------------------------------------------------------
>
> Key: KAFKA-8180
> URL: https://issues.apache.org/jira/browse/KAFKA-8180
> Project: Kafka
> Issue Type: Bug
> Components: controller
> Reporter: Gwen Shapira
> Priority: Major
>
> Scenario:
> - Create large number of topics (In my experiment: 400 topics with 12
> partitions each )
> - Use the admin client to delete all of them in a single batch operation
> - Try to bounce another broker while this is going on
> As you can see from the logs and metrics - topic deletion happens
> synchronously in the controller and it does not do anything else (leader
> elections for instance) while it is busy deleting (which can take many
> minutes for large batches).
> I recommend fixing it by throttling the deletes - no matter how large a batch
> the client sent, the controller should delete a subset and complete a full
> cycle before deleting the next subset.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
