Hi, Kafka, We have a 2 broker staging cluster and a 6 broker prod cluster (all running 0.8.2.1). I'm trying to make sense of the producer performances on both.
Here are some perf test results: ----------------- Prod: kafka-producer-perf-test.sh --broker-list kafkaProd-1:9092,kafkaProd-2:9092,kafkaProd-3:9092,kafkaProd-4:9092,kafkaProd-5:9092,kafkaProd-6:9092 --topics perf_test --message-size 2400 --messages 50000 --batch-size 200 --request-num-acks 1 start.time, end.time, compression, message.size, batch.size, total.data.sent.in.MB, MB.sec, total.data.sent.in.nMsg, nMsg.sec 2016-07-28 15:18:42:093, 2016-07-28 15:18:54:607, 0, 2400, 200, 114.44, 9.1450, 50000, 3995.5250 --------------- Staging: kafka-producer-perf-test.sh --broker-list kafkaStaging-1:9092,kafkaStaging-2:9092 --topics perf_test --message-size 2400 --messages 50000 --batch-size 200 --request-num-acks 1 start.time, end.time, compression, message.size, batch.size, total.data.sent.in.MB, MB.sec, total.data.sent.in.nMsg, nMsg.sec 2016-07-28 14:54:26:310, 2016-07-28 14:56:55:402, 0, 2400, 200, 114.44, 0.7676, 50000, 335.3634 --------------- That is a one magnitude difference in terms of nMsg/sec. Both clusters are balanced. Does these numbers look right? The throughput on the staging cluster looks quite low. Thanks, David
