Hi,
I am now in the process of deciding partitions and replicas for my cluster.
I am making use of perf test utilities and it really helps a lot.
Just measure perf by creating multiple topics with same number of recodrs
with diff partitions and replicas.
Then compare the througput and also look at
Thanks, it helps a lot.
Peter Bukowinski 于2020年2月28日周五 上午5:18写道：
> No, it’s not bad. Kafka is designed to serve data to many consumers at the
> same time, whether they are independent of each other or in the same
> consumer group.
>
> I would encourage you to play with different partition counts
No, it’s not bad. Kafka is designed to serve data to many consumers at the same
time, whether they are independent of each other or in the same consumer group.
I would encourage you to play with different partition counts and use kafka’s
performance testing tools (kafka-producer-perf-test.sh and
I believe no matter the partition count exceeds the broker count, we can
always have the same number of consumer instances as the partition count.
So what I want to know is when two partition exists on the same broker, two
consumer instances will be talking to same broker, is that bad ?
张祥 于2020
The effect for producers isn’t very significant once your topic partition count
exceeds your broker count. For consumers — especially if you are using consumer
groups — the more partitions you have, the more consumer instances you can have
in a single consumer group. (The maximum number of activ
Thanks. What influence does it have for consumers and producers when
partition number is more than broker number, which means at least one
broker serves two partitions for one topic ? performance wise.
Peter Bukowinski 于2020年2月26日周三 下午11:02写道：
> Disk usage is one reason to expand. Another reason
Disk usage is one reason to expand. Another reason is if you need more ingest
or output throughout for your topic data. If your producers aren’t able to send
data to kafka fast enough or your consumers are lagging, you might benefit from
more brokers and more partitions.
-- Peter
> On Feb 26,
In documentation, it is described how to expand cluster:
https://kafka.apache.org/20/documentation.html#basic_ops_cluster_expansion.
But I am wondering what the criteria for expand is. I can only think of
disk usage threshold. For example, suppose several disk usage exceed 80%.
Is this correct and
