There were actually several patches against trunk since 0.8.1.1 that may impact latency however, especially when using acks=-1. So those results in the blog may be a bit better than what you would see in 0.8.1.1.
-Jay On Wed, Jun 18, 2014 at 7:58 PM, Supun Kamburugamuva <supu...@gmail.com> wrote: > My machine configuration is not very high. The average one way latency we > observe is around 10 ~ 15 ms for 50k messages. The outliers doesn't occur > for small messages. For small messages we observe around 6 ms latency. > > Thanks, > Supun.. > > > On Wed, Jun 18, 2014 at 10:18 PM, Neha Narkhede <neha.narkh...@gmail.com> > wrote: > > > what are the latency numbers you observed, avg as well as worst case? > Here > > is a blog that we did recently which should reflect latest performance > > metrics for latency - > > > > > http://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines > > > > > > On Wed, Jun 18, 2014 at 11:01 AM, Supun Kamburugamuva <supu...@gmail.com > > > > wrote: > > > > > I've found this performance test. > > > > > > http://blog.liveramp.com/2013/04/08/kafka-0-8-producer-performance-2/ > > > > > > This performance test has mentioned about the same issue at the end. > > > > > > Thanks, > > > Supun.. > > > > > > > > > On Wed, Jun 18, 2014 at 12:43 PM, Supun Kamburugamuva < > supu...@gmail.com > > > > > > wrote: > > > > > > > The spikes happens without any correlation with the > > > log.flush.interval.message. > > > > They happen more frequently. > > > > > > > > I'm using the latest version. I'm sending the messages to Kafka, then > > > > there is a message receiver, it sends the same messages back through > > > kafka > > > > to original sender. The round trip latency is measured. > > > > > > > > Thanks, > > > > Supun.. > > > > > > > > > > > > On Wed, Jun 18, 2014 at 12:02 PM, Neha Narkhede < > > neha.narkh...@gmail.com > > > > > > > > wrote: > > > > > > > >> Which version of Kafka did you use? When you say latency, do you > mean > > > the > > > >> latency between the producer and consumer? If so, are you using a > > > >> timestamp > > > >> within the message to compute this latency? > > > >> > > > >> > > > >> On Wed, Jun 18, 2014 at 8:15 AM, Magnus Edenhill < > mag...@edenhill.se> > > > >> wrote: > > > >> > > > >> > Hi, > > > >> > > > > >> > do these spikes happen to correlate with > log.flush.interval.messages > > > or > > > >> > log.flush.interval.ms? > > > >> > If so it's the file system sync blockage you are seeing. > > > >> > > > > >> > /Magnus > > > >> > > > > >> > > > > >> > 2014-06-18 16:31 GMT+02:00 Supun Kamburugamuva <supu...@gmail.com > >: > > > >> > > > > >> > > Hi, > > > >> > > > > > >> > > We are trying to evaluate Kafka for a real time application. We > > are > > > >> > sending > > > >> > > 50 Kb messages at a fixed rate. The normal messages have a > > > reasonable > > > >> > > latency. But then there are these outliers that takes > > unpredictable > > > >> > amount > > > >> > > of time. This causes the average latency to increase > dramatically. > > > We > > > >> are > > > >> > > running with basically the default configuration. Any > suggestions > > > for > > > >> > > improving the latency? > > > >> > > > > > >> > > Thanks in advance, > > > >> > > Supun.. > > > >> > > > > > >> > > -- > > > >> > > Supun Kamburugamuva > > > >> > > Member, Apache Software Foundation; http://www.apache.org > > > >> > > E-mail: supu...@gmail.com; Mobile: +1 812 369 6762 > > > >> > > Blog: http://supunk.blogspot.com > > > >> > > > > > >> > > > > >> > > > > > > > > > > > > > > > > -- > > > > Supun Kamburugamuva > > > > Member, Apache Software Foundation; http://www.apache.org > > > > E-mail: supu...@gmail.com; Mobile: +1 812 369 6762 > > > > Blog: http://supunk.blogspot.com > > > > > > > > > > > > > > > > > -- > > > Supun Kamburugamuva > > > Member, Apache Software Foundation; http://www.apache.org > > > E-mail: supu...@gmail.com; Mobile: +1 812 369 6762 > > > Blog: http://supunk.blogspot.com > > > > > > > > > -- > Supun Kamburugamuva > Member, Apache Software Foundation; http://www.apache.org > E-mail: supu...@gmail.com; Mobile: +1 812 369 6762 > Blog: http://supunk.blogspot.com >
