And here is another question. If I load the DStream from database every time I start the job, will the data be loaded when the job is failed and auto restart? If so, both the checkpoint data and database data are loaded, won't this a problem?
Bin Wang <wbi...@gmail.com>于2015年9月16日周三 下午8:40写道： > Will StreamingContex.getOrCreate do this work？What kind of code change > will make it cannot load？ > > Akhil Das <ak...@sigmoidanalytics.com>于2015年9月16日周三 20:20写道： > >> You can't really recover from checkpoint if you alter the code. A better >> approach would be to use some sort of external storage (like a db or >> zookeeper etc) to keep the state (the indexes etc) and then when you deploy >> new code they can be easily recovered. >> >> Thanks >> Best Regards >> >> On Wed, Sep 16, 2015 at 3:52 PM, Bin Wang <wbi...@gmail.com> wrote: >> >>> I'd like to know if there is a way to recovery dstream from checkpoint. >>> >>> Because I stores state in DStream, I'd like the state to be recovered >>> when I restart the application and deploy new code. >>> >> >>
