This is a consumer config: fetch.wait.max.ms
On Tue, Oct 21, 2014 at 11:39 AM, Mohit Anchlia <mohitanch...@gmail.com> wrote: > Is this a parameter I need to set it in kafka server or on the client side? > Also, can you help point out which one exactly is consumer max wait time > from this list? > > https://kafka.apache.org/08/configuration.html > > On Tue, Oct 21, 2014 at 11:35 AM, Jay Kreps <jay.kr...@gmail.com> wrote: > > > There was a bug that could lead to the fetch request from the consumer > > hitting it's timeout instead of being immediately triggered by the > produce > > request. To see if you are effected by that set you consumer max wait > time > > to 1 ms and see if the latency drops to 1 ms (or, alternately, try with > > trunk and see if that fixes the problem). > > > > The reason I suspect this problem is because the default timeout in the > > java consumer is 100ms. > > > > -Jay > > > > On Tue, Oct 21, 2014 at 11:06 AM, Mohit Anchlia <mohitanch...@gmail.com> > > wrote: > > > > > This is the version I am using: kafka_2.10-0.8.1.1 > > > > > > I think this is fairly recent version > > > On Tue, Oct 21, 2014 at 10:57 AM, Jay Kreps <jay.kr...@gmail.com> > wrote: > > > > > > > What version of Kafka is this? Can you try the same test against > trunk? > > > We > > > > fixed a couple of latency related bugs which may be the cause. > > > > > > > > -Jay > > > > > > > > On Tue, Oct 21, 2014 at 10:50 AM, Mohit Anchlia < > > mohitanch...@gmail.com> > > > > wrote: > > > > > > > > > It's consistently close to 100ms which makes me believe that there > > are > > > > some > > > > > settings that I might have to tweak, however, I am not sure how to > > > > confirm > > > > > that assumption :) > > > > > On Tue, Oct 21, 2014 at 8:53 AM, Mohit Anchlia < > > mohitanch...@gmail.com > > > > > > > > > wrote: > > > > > > > > > > > I have a java test that produces messages and then consumer > > consumers > > > > it. > > > > > > Consumers are active all the time. There is 1 consumer for 1 > > > producer. > > > > I > > > > > am > > > > > > measuring the time between the message is successfully written to > > the > > > > > queue > > > > > > and the time consumer picks it up. > > > > > > > > > > > > On Tue, Oct 21, 2014 at 8:32 AM, Neha Narkhede < > > > > neha.narkh...@gmail.com> > > > > > > wrote: > > > > > > > > > > > >> Can you give more information about the performance test? Which > > > test? > > > > > >> Which > > > > > >> queue? How did you measure the dequeue latency. > > > > > >> > > > > > >> On Mon, Oct 20, 2014 at 5:09 PM, Mohit Anchlia < > > > > mohitanch...@gmail.com> > > > > > >> wrote: > > > > > >> > > > > > >> > I am running a performance test and from what I am seeing is > > that > > > > > >> messages > > > > > >> > are taking about 100ms to pop from the queue itself and hence > > > making > > > > > the > > > > > >> > test slow. I am looking for pointers of how I can troubleshoot > > > this > > > > > >> issue. > > > > > >> > > > > > > >> > There seems to be plenty of CPU and IO available. I am running > > 22 > > > > > >> producers > > > > > >> > and 22 consumers in the same group. > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > -- -- Guozhang