Re: Restart App and consume from checkpoint using direct kafka API

2016-06-28 Thread vimal dinakaran
I have implemented the above approach with cassandra db. Thank you all. On Thu, Mar 31, 2016 at 8:26 PM, Cody Koeninger wrote: > Long story short, no. Don't rely on checkpoints if you cant handle > reprocessing some of your data. > > On Thu, Mar 31, 2016 at 3:02 AM, Imre Nagi > wrote: > > I'm

Re: Restart App and consume from checkpoint using direct kafka API

2016-03-31 Thread Cody Koeninger
Long story short, no. Don't rely on checkpoints if you cant handle reprocessing some of your data. On Thu, Mar 31, 2016 at 3:02 AM, Imre Nagi wrote: > I'm dont know how to read the data from the checkpoint. But AFAIK and based > on my experience, I think the best thing that you can do is storing

Re: Restart App and consume from checkpoint using direct kafka API

2016-03-31 Thread Imre Nagi
I'm dont know how to read the data from the checkpoint. But AFAIK and based on my experience, I think the best thing that you can do is storing the offset to a particular storage such as database everytime you consume the message. Then read the offset from the database everytime you want to start r

Restart App and consume from checkpoint using direct kafka API

2016-03-31 Thread vimal dinakaran
Hi, In the blog https://github.com/koeninger/kafka-exactly-once/blob/master/blogpost.md It is mentioned that enabling checkpoint works as long as the app jar is unchanged. If I want to upgrade the jar with the latest code and consume from kafka where it was stopped , how to do that ? Is there a