[
https://issues.apache.org/jira/browse/KAFKA-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Bansal updated KAFKA-9063:
--
Description:
5 broker cluster
Topic partitions = 1
Replication factor = 3
Ac mode = all
Send type = Asynchronous
Message size = 100 bytes
Log compaction = Enabled
# We started throughput test with 1 topic and number of topics present in
cluster at that time were 1 (excluding already existing topics 1000)
# We left it to run for about 2 hours (The mean throughput we observed during
this period was 54k msgs/sec)
# After this, we started creating 10,000 topics one by one using a script
# We noted throughput values after creating 100 topics and after
200,300,400…so on till 10,000 were created
# After all 10,000 topics were created we left test to run for another 1 hr.
During the entire duration, we were producing only on a single topic.
What could be causing producer throughput drop by just adding number of topics?
!image-2019-10-18-10-22-40-372.png!
was:
5 broker cluster
Topic partitions = 1
Replication factor = 3
Ac mode = all
Send type = Asynchronous
Message size = 100 bytes
Log compaction = Enabled
# We started throughput test with 1 topic and number of topics present in
cluster at that time were 1 (excluding already existing topics 1000)
# We left it to run for about 2 hours (The mean throughput we observed during
this period was 54k msgs/sec)
# After this, we started creating 10,000 topics one by one using a script
# We noted throughput values after creating 100 topics and after
200,300,400…so on till 10,000 were created
# After all 10,000 topics were created we left test to run for another 1 hr.
During the entire duration, we were producing only on a single topic.
What could be causing performance drop
!image-2019-10-18-10-22-40-372.png!
> KAfka producer throughput drops with number of topics even when producer is
> producing on one topic
> --
>
> Key: KAFKA-9063
> URL: https://issues.apache.org/jira/browse/KAFKA-9063
> Project: Kafka
> Issue Type: Improvement
>Affects Versions: 2.2.1
>Reporter: Pradeep Bansal
>Priority: Major
> Attachments: image-2019-10-18-10-22-40-372.png
>
>
> 5 broker cluster
> Topic partitions = 1
> Replication factor = 3
> Ac mode = all
> Send type = Asynchronous
> Message size = 100 bytes
> Log compaction = Enabled
> # We started throughput test with 1 topic and number of topics present in
> cluster at that time were 1 (excluding already existing topics 1000)
> # We left it to run for about 2 hours (The mean throughput we observed
> during this period was 54k msgs/sec)
> # After this, we started creating 10,000 topics one by one using a script
> # We noted throughput values after creating 100 topics and after
> 200,300,400…so on till 10,000 were created
> # After all 10,000 topics were created we left test to run for another 1 hr.
> During the entire duration, we were producing only on a single topic.
>
> What could be causing producer throughput drop by just adding number of
> topics?
> !image-2019-10-18-10-22-40-372.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
