Exactly once Kafka Sink Connectors typically store the offset externally in the 
same atomic write as they store the messages. That way after a crash, they can 
check the external store (HSFS, JSDC, etc) retrieve the last committed offset 
and seek the the next message and continue processing with no duplicates and 
exactly once semantics.

-hans




> On Dec 21, 2016, at 4:39 PM, kant kodali <[email protected]> wrote:
> 
> How does Kafka emulate exactly once processing currently? Does it require
> the producer to send at least once and consumer to de dupe?
> 
> I did do my research but I feel like I am going all over the place so a
> simple short answer would be great!
> 
> Thanks!

Reply via email to