[ https://issues.apache.org/jira/browse/KAFKA-9590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gokul Ramanan Subramanian reassigned KAFKA-9590: ------------------------------------------------ Assignee: Gokul Ramanan Subramanian > Add read-only configuration to limit number of topic partitions > --------------------------------------------------------------- > > Key: KAFKA-9590 > URL: https://issues.apache.org/jira/browse/KAFKA-9590 > Project: Kafka > Issue Type: Improvement > Components: admin > Reporter: Gokul Ramanan Subramanian > Assignee: Gokul Ramanan Subramanian > Priority: Minor > > Currently, a Kafka cluster does not limit the number of topic partitions > despite the fact that on a given system, having more than a certain number of > topic partitions (without even any messages written / read) renders the > cluster unusable. > > The ask here is to have a read-only Kafka configuration named something like > *max.broker.partitions* (an integer, which applies at the broker level and > defaults to INT.MAX) that the administrator the cluster can set to ensure > smooth operation. > > Things to keep in mind: > # If a broker already has more than *max.broker.partitions* number of > partitions at the time when the *max.broker.partitions* configuration is > first applied on the broker, the broker should disallow any more topic > partitions. This allows for graceful reassignments and in-place upgrade to a > Kafka version that supports the *max.broker.partitions* configuration. > # Different brokers can technically have different *max.broker.partitions* > in case of a heterogenous cluster, where not all broker nodes operate with > the same system limits (such as CPU, memory etc.) > # When creating a topic partition, the controller has to identify brokers > which will replicate the topic partition. Currently, the controller > distributes the leadership of the topic partitions to ensure uniform load. > This algorithm will need to account for the *max.broker.partitions* > configuration. > -- This message was sent by Atlassian Jira (v8.3.4#803005)