Don't you have the same problem using SimpleConsumer? How does another process know a SimpleConsumer hangs?
Thanks, Jun On Mon, Nov 10, 2014 at 9:47 AM, Srinivas Reddy Kancharla < getre...@gmail.com> wrote: > Thanks Jun for your response. > Here is my scenario: > > topicCountMap.put(topic, new Integer(2)); > Map<String, List<KafkaStream<byte[], byte[]>>> consumerMap = > consumer.createMessageStreams(topicCountMap); > List<KafkaStream<byte[], byte[]>> streams = consumerMap.get(topic); > > So from above scenario (only 1 partition) , there will be 2 threads C1 and > C2, and one of the thread would be holding one partition. In my scenario, > C1 would be hung after consuming the message from stream and in hung state > during processing of message. So in such scenario, looks like C2 will not > get hold of partition stream unless C1 didn't die.. > > So considering such scenarios, I am planning to use SimpleConsumer where I > will have more control on consuming Partition content. But I have to > maintain offset state in zookeeper. Let me know if this approach is correct > for such hung scenarios. > > Thanks and regards, > Srini > > > On Sun, Nov 9, 2014 at 9:12 PM, Jun Rao <jun...@gmail.com> wrote: > > > If C1 dies, C2 will be owning that partition. However, C1 has to really > > die, which typically means that either you close the consumer connector > or > > the jvm of C1 is gone. > > > > In your case, it seems that C1 didn't die, it just hung. Do you know why > C1 > > hung? > > > > Thanks, > > > > Jun > > > > On Fri, Nov 7, 2014 at 3:34 PM, Srinivas Reddy Kancharla < > > getre...@gmail.com > > > wrote: > > > > > Hi, > > > > > > I have a scenario where I have 1 partition and 1 consumer group having > 2 > > > consumer threads running say C1 and C2. Since there is only one > partition > > > for a given topic, say C1 is holding that partition. Now due to some > > reason > > > if C1 dies, can C2 get hold of that partition? > > > > > > i.e. C1 was busy with KafkaStream instance, for any reason if C1 dies > or > > in > > > hung state, Can we make C2 talking to KafkaStream (for Partition 0). > > > I am facing this issue where I have 10 messages in partition 0 and C1 > was > > > consuming it. At message 4, C1 went into hung state. Now I would like > to > > > make C2 to consumer other messages which are not consumed by C1. > > > > > > Thank and regards, > > > Srini > > > > > >
