There were actually several patches against trunk since 0.8.1.1 that may
impact latency however, especially when using acks=-1. So those results in
the blog may be a bit better than what you would see in 0.8.1.1.
-Jay
On Wed, Jun 18, 2014 at 7:58 PM, Supun Kamburugamuva
wrote:
> My machine con
My machine configuration is not very high. The average one way latency we
observe is around 10 ~ 15 ms for 50k messages. The outliers doesn't occur
for small messages. For small messages we observe around 6 ms latency.
Thanks,
Supun..
On Wed, Jun 18, 2014 at 10:18 PM, Neha Narkhede
wrote:
> wh
what are the latency numbers you observed, avg as well as worst case? Here
is a blog that we did recently which should reflect latest performance
metrics for latency -
http://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines
On Wed, Jun 18, 2014
I've found this performance test.
http://blog.liveramp.com/2013/04/08/kafka-0-8-producer-performance-2/
This performance test has mentioned about the same issue at the end.
Thanks,
Supun..
On Wed, Jun 18, 2014 at 12:43 PM, Supun Kamburugamuva
wrote:
> The spikes happens without any correlati
The spikes happens without any correlation with the
log.flush.interval.message.
They happen more frequently.
I'm using the latest version. I'm sending the messages to Kafka, then there
is a message receiver, it sends the same messages back through kafka to
original sender. The round trip latency i
Which version of Kafka did you use? When you say latency, do you mean the
latency between the producer and consumer? If so, are you using a timestamp
within the message to compute this latency?
On Wed, Jun 18, 2014 at 8:15 AM, Magnus Edenhill wrote:
> Hi,
>
> do these spikes happen to correlate
Hi,
do these spikes happen to correlate with log.flush.interval.messages or
log.flush.interval.ms?
If so it's the file system sync blockage you are seeing.
/Magnus
2014-06-18 16:31 GMT+02:00 Supun Kamburugamuva :
> Hi,
>
> We are trying to evaluate Kafka for a real time application. We are sen
Hi,
We are trying to evaluate Kafka for a real time application. We are sending
50 Kb messages at a fixed rate. The normal messages have a reasonable
latency. But then there are these outliers that takes unpredictable amount
of time. This causes the average latency to increase dramatically. We are
