mjaschob-twilio opened a new pull request #11277: URL: https://github.com/apache/kafka/pull/11277
When new topics are created through the admin interface, the Kafka controller creates the partition assignments for these topics according to the algorithm defined in `AdminUtils.assignReplicasToBrokers`. All (alive) brokers in the cluster become candidates for hosting partitions. However, sometimes cluster administrators don't want certain brokers to host partitions for newly created topics. This can be for a variety of reasons, e.g.: - a broker may be on a host that is slated for retirement and about to be shut down - a broker may be acting as a canary of a new Kafka version and is being tested with a controlled set of topic-partitions, and should be restricted from hosting any other topics - similarly, a broker may in use for administrative operations, such as dumping log segments, and consequently may be unstable and ideally as little-used as possible One solution is for the cluster admins to not give out CREATE permissions to clients, and instead provide a service/abstraction for creating new topics with explicit partition assignments. Unfortunately, in a world with Kafka Streams applications, this is too restrictive: Kafka Streams clients expect to use the Kafka tooling to create topics when their topology changes, or when they need to reset their apps. Since there is currently there is no way to control which brokers are eligible to be assigned newly created partitions, I am proposing a new configuration parameter, `create.topic.broker.filter.policy.class.name` to allow cluster administrators to provide a pluggable class to control whether a broker should be allowed to host new partitions. The class implements a simple interface to give a binary yes/no verdict on each broker: ```java boolean isAllowedToHostPartitions(Broker broker); ``` This follows a pattern similar to `create.topic.policy.class.name` and `alter.topic.policy.class.name`, which also provide cluster-level control over other aspects of topic creation. This contribution is my original work and I license the work to the project under the project's open source license. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
