Thanks Raghu..I know this is not Kafka forum ,but could Kafka be sending
redundant records?I need to set a counter outside of my code in Redis and check
it.Thanks
From: Raghu Angadi <[email protected]>
To: [email protected]; amir bahmanyari <[email protected]>
Sent: Tuesday, July 26, 2016 1:28 PM
Subject: Re: Avoid reading duplicates from KafkaIO!!!
On Tue, Jul 26, 2016 at 1:13 PM, amir bahmanyari <[email protected]> wrote:
Could this be a possibility with KafkaIO function & how it interacts with Kafka
server?
No. Not expected.It should be a bug if observe duplicates. did you have
autoscaling enabled (not that I expect a bug here)? Did the number of workers
change?
If so, how can I prevent KafkaIO from duplicate readings?
