Re: kafka sending duplicate content to consumer

2015-01-23 Thread Tousif
Thanks, i'm a newbie wrt kafka. im using kafka-spout. here is the fail handler of kafka-spout, so to avoid replaying do i need to remove below snipped from fail handler?. Can you point me to official kafka-spout. can i use the one provided under external folder. if (_failHandler.shouldReplay(id

Re: kafka sending duplicate content to consumer

2015-01-23 Thread svante karlsson
A kafka broker never pushes data to a consumer. It's the consumer that does a long fetch and it provides the offset to read from. The problem lies in how your consumer handles the for example 1000 messages that it just got. If you handle 500 of them and crash without committing the offsets somewhe

kafka sending duplicate content to consumer

2015-01-22 Thread Tousif
Hi, i want know in which situation does kafka send same event multiple times to consumer. Is there a consumer side configuration to tell kafka to send only once and stop retries? -- Regards Tousif Khazi