Re: multiple producer throughput
The message size is 100 bytes and each producer sends out 50million messages. It's the number used by the "benchmarking kafka" post. http://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines Thanks. On Mon, Jul 27, 2015 at 4:15 PM, Prabhjot Bharaj wrote: > Hi, > > Have you tried with acks=1 and -1 as well? > Please share the numbers and the message size > > Regards, > Prabcs > On Jul 27, 2015 10:24 PM, "Yuheng Du" wrote: > > > Hi, > > > > I am running 40 producers on 40 nodes cluster. The messages are sent to 6 > > brokers in another cluster. The producers are running ProducerPerformance > > test. > > > > When 20 nodes are running, the throughput is around 13MB/s and when > running > > 40 nodes, the throughput is around 9MB/s. > > > > I have set log.retention.ms=9000 to delete the unwanted messages, just > to > > avoid the disk space to be filled. > > > > So I want to know how should I tune the system to get better throughput > > result? Thanks. > > >
Re: multiple producer throughput
Hi, Have you tried with acks=1 and -1 as well? Please share the numbers and the message size Regards, Prabcs On Jul 27, 2015 10:24 PM, "Yuheng Du" wrote: > Hi, > > I am running 40 producers on 40 nodes cluster. The messages are sent to 6 > brokers in another cluster. The producers are running ProducerPerformance > test. > > When 20 nodes are running, the throughput is around 13MB/s and when running > 40 nodes, the throughput is around 9MB/s. > > I have set log.retention.ms=9000 to delete the unwanted messages, just to > avoid the disk space to be filled. > > So I want to know how should I tune the system to get better throughput > result? Thanks. >
multiple producer throughput
Hi, I am running 40 producers on 40 nodes cluster. The messages are sent to 6 brokers in another cluster. The producers are running ProducerPerformance test. When 20 nodes are running, the throughput is around 13MB/s and when running 40 nodes, the throughput is around 9MB/s. I have set log.retention.ms=9000 to delete the unwanted messages, just to avoid the disk space to be filled. So I want to know how should I tune the system to get better throughput result? Thanks.
