Re: Restart App and consume from checkpoint using direct kafka API

2016-06-28 Thread vimal dinakaran
I have implemented the above approach with cassandra db. Thank you all. On Thu, Mar 31, 2016 at 8:26 PM, Cody Koeninger wrote: > Long story short, no. Don't rely on checkpoints if you cant handle > reprocessing some of your data. > > On Thu, Mar 31, 2016 at 3:02 AM, Imre

Re: Restart App and consume from checkpoint using direct kafka API

2016-03-31 Thread Cody Koeninger
Long story short, no. Don't rely on checkpoints if you cant handle reprocessing some of your data. On Thu, Mar 31, 2016 at 3:02 AM, Imre Nagi wrote: > I'm dont know how to read the data from the checkpoint. But AFAIK and based > on my experience, I think the best thing

Re: Restart App and consume from checkpoint using direct kafka API

2016-03-31 Thread Imre Nagi
I'm dont know how to read the data from the checkpoint. But AFAIK and based on my experience, I think the best thing that you can do is storing the offset to a particular storage such as database everytime you consume the message. Then read the offset from the database everytime you want to start

Restart App and consume from checkpoint using direct kafka API

2016-03-31 Thread vimal dinakaran
Hi, In the blog https://github.com/koeninger/kafka-exactly-once/blob/master/blogpost.md It is mentioned that enabling checkpoint works as long as the app jar is unchanged. If I want to upgrade the jar with the latest code and consume from kafka where it was stopped , how to do that ? Is there a