As @Milind said, it is possible that a consumer consumes the same message more than once. This happens when there is an unclean shutdown of the consumer and it is not able to commit its latest offset to Zookeeper. When the failed consumer comes up, it would fetch the stale offset from zookeeper thus re-consuming a small set of messages.
On Wed, Jul 31, 2013 at 3:31 PM, Milind Parikh <milindpar...@gmail.com>wrote: > It is possible to consume the same message more than once with the same > consumer. However WHAT you actually do with the message (such as idempotent > writes) is the tricker part. > > Regards > Milind > > > > On Wed, Jul 31, 2013 at 8:22 AM, Oleg Ruchovets <oruchov...@gmail.com > >wrote: > > > Hi , > > > > I just don't know which mail list is correct to post this question( storm > > or kafka)? Sorry for cross post. > > > > I just read the documentation which describe guaranteed message > > processing with storm - > > https://github.com/nathanmarz/storm/wiki/Guaranteeing-message-processing > . > > > > The question actually what will be with the message which was consumed by > > storm and it is failed to process. In case I'll use anchoring technique , > > trying to process the message the second time: will this be available in > > kafka ( I am using storm-kafka spout)? > > > > I mean Is it possible to consume the same message in kafka more then one > > time with the same consumer? > > > > Thanks > > Oleg. > > >
