My machine configuration is not very high. The average one way latency we observe is around 10 ~ 15 ms for 50k messages. The outliers doesn't occur for small messages. For small messages we observe around 6 ms latency.
Thanks, Supun.. On Wed, Jun 18, 2014 at 10:18 PM, Neha Narkhede <neha.narkh...@gmail.com> wrote: > what are the latency numbers you observed, avg as well as worst case? Here > is a blog that we did recently which should reflect latest performance > metrics for latency - > > http://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines > > > On Wed, Jun 18, 2014 at 11:01 AM, Supun Kamburugamuva <supu...@gmail.com> > wrote: > > > I've found this performance test. > > > > http://blog.liveramp.com/2013/04/08/kafka-0-8-producer-performance-2/ > > > > This performance test has mentioned about the same issue at the end. > > > > Thanks, > > Supun.. > > > > > > On Wed, Jun 18, 2014 at 12:43 PM, Supun Kamburugamuva <supu...@gmail.com > > > > wrote: > > > > > The spikes happens without any correlation with the > > log.flush.interval.message. > > > They happen more frequently. > > > > > > I'm using the latest version. I'm sending the messages to Kafka, then > > > there is a message receiver, it sends the same messages back through > > kafka > > > to original sender. The round trip latency is measured. > > > > > > Thanks, > > > Supun.. > > > > > > > > > On Wed, Jun 18, 2014 at 12:02 PM, Neha Narkhede < > neha.narkh...@gmail.com > > > > > > wrote: > > > > > >> Which version of Kafka did you use? When you say latency, do you mean > > the > > >> latency between the producer and consumer? If so, are you using a > > >> timestamp > > >> within the message to compute this latency? > > >> > > >> > > >> On Wed, Jun 18, 2014 at 8:15 AM, Magnus Edenhill <mag...@edenhill.se> > > >> wrote: > > >> > > >> > Hi, > > >> > > > >> > do these spikes happen to correlate with log.flush.interval.messages > > or > > >> > log.flush.interval.ms? > > >> > If so it's the file system sync blockage you are seeing. > > >> > > > >> > /Magnus > > >> > > > >> > > > >> > 2014-06-18 16:31 GMT+02:00 Supun Kamburugamuva <supu...@gmail.com>: > > >> > > > >> > > Hi, > > >> > > > > >> > > We are trying to evaluate Kafka for a real time application. We > are > > >> > sending > > >> > > 50 Kb messages at a fixed rate. The normal messages have a > > reasonable > > >> > > latency. But then there are these outliers that takes > unpredictable > > >> > amount > > >> > > of time. This causes the average latency to increase dramatically. > > We > > >> are > > >> > > running with basically the default configuration. Any suggestions > > for > > >> > > improving the latency? > > >> > > > > >> > > Thanks in advance, > > >> > > Supun.. > > >> > > > > >> > > -- > > >> > > Supun Kamburugamuva > > >> > > Member, Apache Software Foundation; http://www.apache.org > > >> > > E-mail: supu...@gmail.com; Mobile: +1 812 369 6762 > > >> > > Blog: http://supunk.blogspot.com > > >> > > > > >> > > > >> > > > > > > > > > > > > -- > > > Supun Kamburugamuva > > > Member, Apache Software Foundation; http://www.apache.org > > > E-mail: supu...@gmail.com; Mobile: +1 812 369 6762 > > > Blog: http://supunk.blogspot.com > > > > > > > > > > > > -- > > Supun Kamburugamuva > > Member, Apache Software Foundation; http://www.apache.org > > E-mail: supu...@gmail.com; Mobile: +1 812 369 6762 > > Blog: http://supunk.blogspot.com > > > -- Supun Kamburugamuva Member, Apache Software Foundation; http://www.apache.org E-mail: supu...@gmail.com; Mobile: +1 812 369 6762 Blog: http://supunk.blogspot.com
