Your batch.size is 8196 and your buffer.memory is 67108864. This means 67108864/8196 ~ 8188 batches are in memory ready to the sent. There is only one thread io thread sending them. I would guess that the io thread ( kafka-producer-network-thread) would be busy. Please check it in visual vm.
In steady state, every batch has to wait for the previous 8187 batches to be done before it gets a chance to be sent out, but the latency is counted from the time is added to the queue. This is the reason that you are seeing very high end-to-end latency. Have the buffer.memory to be only twice that of the batch.size so that while one is in flight, you can another batch ready to go (and the KafkaProducer would block to send more when there is no memory and this way the batches are not waiting in the queue unnecessarily) . Also may be you want to increase the batch.size further more, you will get even better throughput with more or less same latency (as there is no shortage of events in the test program). On Thu, Aug 13, 2015 at 1:13 PM Yuheng Du <yuheng.du.h...@gmail.com> wrote: > Yes there is. But if we are using ProducerPerformance test, it's configured > as giving input when running the test command. Do you write a java program > to test the latency? Thanks. > > On Thu, Aug 13, 2015 at 3:54 PM, Alvaro Gareppe <agare...@gmail.com> > wrote: > > > I'm using last one, but not using the ProducerPerformance, I created my > > own. but I think there is a producer.properties file in config folder in > > kafka.. is that configuration not for this tester ? > > > > On Thu, Aug 13, 2015 at 4:18 PM, Yuheng Du <yuheng.du.h...@gmail.com> > > wrote: > > > > > Thank you Alvaro, > > > > > > How to use sync producers? I am running the standard > ProducerPerformance > > > test from kafka to measure the latency of each message to send from > > > producer to broker only. > > > The command is like "bin/kafka-run-class.sh > > > org.apache.kafka.clients.tools.ProducerPerformance test7 50000000 100 > -1 > > > acks=1 bootstrap.servers=esv4-hcl198.grid.linkedin.com:9092 > > > buffer.memory=67108864 batch.size=8196" > > > > > > For running producers, where should I put the producer.type=sync > > > configuration into? The config/server.properties? Also Does this mean > we > > > are using batch size of 1? Which version of Kafka are you using? > > > thanks. > > > > > > On Thu, Aug 13, 2015 at 3:01 PM, Alvaro Gareppe <agare...@gmail.com> > > > wrote: > > > > > > > Are you measuring latency as time between producer and consumer ? > > > > > > > > In that case, the ack shouldn't affect the latency, cause even tough > > your > > > > producer is not going to wait for the ack, the consumer will only get > > the > > > > message after its commited in the server. > > > > > > > > About latency my best result occur with sync producers, but the > > > throughput > > > > is much lower in that case. > > > > > > > > About not flushing to disk I'm pretty sure that it's not an option in > > > kafka > > > > (correct me if I'm wrong) > > > > > > > > Regards, > > > > Alvaro Gareppe > > > > > > > > On Thu, Aug 13, 2015 at 12:59 PM, Yuheng Du < > yuheng.du.h...@gmail.com> > > > > wrote: > > > > > > > > > Also, the latency results show no major difference when using ack=0 > > or > > > > > ack=1. Why is that? > > > > > > > > > > On Thu, Aug 13, 2015 at 11:51 AM, Yuheng Du < > > yuheng.du.h...@gmail.com> > > > > > wrote: > > > > > > > > > > > I am running an experiment where 92 producers is publishing data > > > into 6 > > > > > > brokers and 10 consumer are reading online data simultaneously. > > > > > > > > > > > > How should I do to reduce the latency? Currently when I run the > > > > producer > > > > > > performance test the average latency is around 10s. > > > > > > > > > > > > Should I disable log.flush? How to do that? Thanks. > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Ing. Alvaro Gareppe > > > > agare...@gmail.com > > > > > > > > > > > > > > > -- > > Ing. Alvaro Gareppe > > agare...@gmail.com > > >
