Then you will need even more parallel producers to saturate a 10 GigE network (if you don't hit you disk I/O limit first)
-hans > On Apr 1, 2017, at 3:15 PM, Archie <anubhavnidhi1...@gmail.com> wrote: > > My replication factor is 1. > > Thanks, > Archie > >> On Sat, Apr 1, 2017 at 3:51 PM, Hans Jespersen <h...@confluent.io> wrote: >> >> What replication factor are you using? If you have a default replication >> factor = 3 then a publish rate of 1.4 Gbps is actually 1.4 Gbps *3 = 4.2 >> Gbps of network traffic. If you are also consuming at the same time then >> it’s actually 4.2 Gbps + 1.4 Gbps = 5.6 Gbps. >> >> You would completely saturate the network if you added a second producer >> and consumer at those rates (if your storage system can keep up to the >> network bandwidth). >> >> -hans >> >> >> >>> On Apr 1, 2017, at 10:25 AM, Archie <anubhavnidhi1...@gmail.com> wrote: >>> >>> I have set up my kafka cluster in a network with 9.3 Gbps links. And am >>> using the bin/kafka-producer-perf-test.sh script to test the throughput >>> performance of kafka for a topic which has 1 partition. >>> >>> Right now I am only able to use 1.4 Gbps of network link, i.e. the script >>> returns a throughput of 1.4 Gbps (179 MBps) >>> >>> My basic question is how can I increase the throughput of kafka to >>> completely saturate the network? >>> >>> Thanks, >>> Archie >> >>
