The effect for producers isn’t very significant once your topic partition count exceeds your broker count. For consumers — especially if you are using consumer groups — the more partitions you have, the more consumer instances you can have in a single consumer group. (The maximum number of active consumers in a consumer group = the total number of topic partitions assigned to the group.)
As long as you are not exceeding the broker’s network and disk IO, your total consumer throughput goes up with more partitions. Additional network and disk IO are a benefit of additional brokers. -- Peter > On Feb 26, 2020, at 10:23 PM, 张祥 <xiangzhang1...@gmail.com> wrote: > > Thanks. What influence does it have for consumers and producers when > partition number is more than broker number, which means at least one > broker serves two partitions for one topic ? performance wise. > > Peter Bukowinski <pmb...@gmail.com> 于2020年2月26日周三 下午11:02写道： > >> Disk usage is one reason to expand. Another reason is if you need more >> ingest or output throughout for your topic data. If your producers aren’t >> able to send data to kafka fast enough or your consumers are lagging, you >> might benefit from more brokers and more partitions. >> >> -- Peter >> >>>> On Feb 26, 2020, at 12:56 AM, 张祥 <xiangzhang1...@gmail.com> wrote: >>> >>> In documentation, it is described how to expand cluster: >>> >> https://kafka.apache.org/20/documentation.html#basic_ops_cluster_expansion >> . >>> But I am wondering what the criteria for expand is. I can only think of >>> disk usage threshold. For example, suppose several disk usage exceed 80%. >>> Is this correct and is there more ? >>
