Kafka consumer picking up the same message multiple times

2016-09-27 Thread Shamik Bandopadhyay
Hi, I've recently started using kafka to read documents coming through a web crawler. What I'm noticing is when I'm dealing with few million documents, the consumer is processing the same message over and over again. Looks like the data is not getting committed for some reason. This is not the c

Re: Problem consuming message using custom serializer

2016-09-09 Thread Shamik Bandopadhyay
Anyone ? On Tue, Sep 6, 2016 at 4:21 PM, Shamik Bandopadhyay wrote: > Hi, > > I'm trying to send java object using kryo object serializer . The > producer is able to send the payload to the queue, but I'm having issues > reading the data in consumer. I'm using co

Problem consuming message using custom serializer

2016-09-06 Thread Shamik Bandopadhyay
Hi, I'm trying to send java object using kryo object serializer . The producer is able to send the payload to the queue, but I'm having issues reading the data in consumer. I'm using consumer group using KafkaStream. The consumer code is based out of the example in kafka documentation. Here's th