By increasing partitions and using kafka from master branch I was able to
cut down the response times into half. But it still seems high and it looks
like there still is a delay between a successful post and the first time
message is seen by the consumers. There are plenty of resources available.
I can't find this property in server.properties file. Is that the right
place to set this parameter?
On Tue, Oct 21, 2014 at 6:27 PM, Jun Rao jun...@gmail.com wrote:
Could you also set replica.fetch.wait.max.ms in the broker to sth much
smaller?
Thanks,
Jun
On Tue, Oct 21, 2014 at 2:15
I have a java test that produces messages and then consumer consumers it.
Consumers are active all the time. There is 1 consumer for 1 producer. I am
measuring the time between the message is successfully written to the queue
and the time consumer picks it up.
On Tue, Oct 21, 2014 at 8:32 AM, Neha
This is the version I am using: kafka_2.10-0.8.1.1
I think this is fairly recent version
On Tue, Oct 21, 2014 at 10:57 AM, Jay Kreps jay.kr...@gmail.com wrote:
What version of Kafka is this? Can you try the same test against trunk? We
fixed a couple of latency related bugs which may be the
There was a bug that could lead to the fetch request from the consumer
hitting it's timeout instead of being immediately triggered by the produce
request. To see if you are effected by that set you consumer max wait time
to 1 ms and see if the latency drops to 1 ms (or, alternately, try with
trunk
Is this a parameter I need to set it in kafka server or on the client side?
Also, can you help point out which one exactly is consumer max wait time
from this list?
https://kafka.apache.org/08/configuration.html
On Tue, Oct 21, 2014 at 11:35 AM, Jay Kreps jay.kr...@gmail.com wrote:
There was a
This is a consumer config:
fetch.wait.max.ms
On Tue, Oct 21, 2014 at 11:39 AM, Mohit Anchlia mohitanch...@gmail.com
wrote:
Is this a parameter I need to set it in kafka server or on the client side?
Also, can you help point out which one exactly is consumer max wait time
from this list?
I set the property to 1 in the consumer code that is passed to
createJavaConsumerConnector
code, but it didn't seem to help
props.put(fetch.wait.max.ms, fetchMaxWait);
On Tue, Oct 21, 2014 at 1:21 PM, Guozhang Wang wangg...@gmail.com wrote:
This is a consumer config:
fetch.wait.max.ms
On
Most of the consumer threads seems to be waiting:
ConsumerFetcherThread-groupA_ip-10-38-19-230-1413925671158-3cc3e22f-0-0
prio=10 tid=0x7f0aa84db800 nid=0x5be9 runnable [0x7f0a5a618000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
