Most of the consumer threads seems to be waiting:
"ConsumerFetcherThread-groupA_ip-10-38-19-230-1413925671158-3cc3e22f-0-0"
prio=10 tid=0x00007f0aa84db800 nid=0x5be9 runnable [0x00007f0a5a618000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
- locked <0x000000009515bec0> (a sun.nio.ch.Util$2)
- locked <0x000000009515bea8> (a
java.util.Collections$UnmodifiableSet)
- locked <0x0000000095511d00> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
at
sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:221)
- locked <0x000000009515bd28> (a java.lang.Object)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
- locked <0x0000000095293828> (a
sun.nio.ch.SocketAdaptor$SocketInputStream)
at
java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
- locked <0x000000009515bcb0> (a java.lang.Object)
at kafka.utils.Utils$.read(Utils.scala:375)
On Tue, Oct 21, 2014 at 2:15 PM, Mohit Anchlia <mohitanch...@gmail.com>
wrote:
> I set the property to 1 in the consumer code that is passed to
> "createJavaConsumerConnector"
> code, but it didn't seem to help
>
> props.put("fetch.wait.max.ms", fetchMaxWait);
>
> On Tue, Oct 21, 2014 at 1:21 PM, Guozhang Wang <wangg...@gmail.com> wrote:
>
>> This is a consumer config:
>>
>> fetch.wait.max.ms
>>
>> On Tue, Oct 21, 2014 at 11:39 AM, Mohit Anchlia <mohitanch...@gmail.com>
>> wrote:
>>
>> > Is this a parameter I need to set it in kafka server or on the client
>> side?
>> > Also, can you help point out which one exactly is consumer max wait time
>> > from this list?
>> >
>> > https://kafka.apache.org/08/configuration.html
>> >
>> > On Tue, Oct 21, 2014 at 11:35 AM, Jay Kreps <jay.kr...@gmail.com>
>> wrote:
>> >
>> > > There was a bug that could lead to the fetch request from the consumer
>> > > hitting it's timeout instead of being immediately triggered by the
>> > produce
>> > > request. To see if you are effected by that set you consumer max wait
>> > time
>> > > to 1 ms and see if the latency drops to 1 ms (or, alternately, try
>> with
>> > > trunk and see if that fixes the problem).
>> > >
>> > > The reason I suspect this problem is because the default timeout in
>> the
>> > > java consumer is 100ms.
>> > >
>> > > -Jay
>> > >
>> > > On Tue, Oct 21, 2014 at 11:06 AM, Mohit Anchlia <
>> mohitanch...@gmail.com>
>> > > wrote:
>> > >
>> > > > This is the version I am using: kafka_2.10-0.8.1.1
>> > > >
>> > > > I think this is fairly recent version
>> > > > On Tue, Oct 21, 2014 at 10:57 AM, Jay Kreps <jay.kr...@gmail.com>
>> > wrote:
>> > > >
>> > > > > What version of Kafka is this? Can you try the same test against
>> > trunk?
>> > > > We
>> > > > > fixed a couple of latency related bugs which may be the cause.
>> > > > >
>> > > > > -Jay
>> > > > >
>> > > > > On Tue, Oct 21, 2014 at 10:50 AM, Mohit Anchlia <
>> > > mohitanch...@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > It's consistently close to 100ms which makes me believe that
>> there
>> > > are
>> > > > > some
>> > > > > > settings that I might have to tweak, however, I am not sure how
>> to
>> > > > > confirm
>> > > > > > that assumption :)
>> > > > > > On Tue, Oct 21, 2014 at 8:53 AM, Mohit Anchlia <
>> > > mohitanch...@gmail.com
>> > > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > > > I have a java test that produces messages and then consumer
>> > > consumers
>> > > > > it.
>> > > > > > > Consumers are active all the time. There is 1 consumer for 1
>> > > > producer.
>> > > > > I
>> > > > > > am
>> > > > > > > measuring the time between the message is successfully
>> written to
>> > > the
>> > > > > > queue
>> > > > > > > and the time consumer picks it up.
>> > > > > > >
>> > > > > > > On Tue, Oct 21, 2014 at 8:32 AM, Neha Narkhede <
>> > > > > neha.narkh...@gmail.com>
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > >> Can you give more information about the performance test?
>> Which
>> > > > test?
>> > > > > > >> Which
>> > > > > > >> queue? How did you measure the dequeue latency.
>> > > > > > >>
>> > > > > > >> On Mon, Oct 20, 2014 at 5:09 PM, Mohit Anchlia <
>> > > > > mohitanch...@gmail.com>
>> > > > > > >> wrote:
>> > > > > > >>
>> > > > > > >> > I am running a performance test and from what I am seeing
>> is
>> > > that
>> > > > > > >> messages
>> > > > > > >> > are taking about 100ms to pop from the queue itself and
>> hence
>> > > > making
>> > > > > > the
>> > > > > > >> > test slow. I am looking for pointers of how I can
>> troubleshoot
>> > > > this
>> > > > > > >> issue.
>> > > > > > >> >
>> > > > > > >> > There seems to be plenty of CPU and IO available. I am
>> running
>> > > 22
>> > > > > > >> producers
>> > > > > > >> > and 22 consumers in the same group.
>> > > > > > >> >
>> > > > > > >>
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>>
>>
>> --
>> -- Guozhang
>>
>
>
