Re: Checkpoint not working after driver restart

2015-11-07 Thread vimal dinakaran
I am pasting the code here . Please let me know if there is any sequence that is wrong. def createContext(checkpointDirectory: String, config: Config): StreamingContext = { println("Creating new context") val conf = new

Checkpoint not working after driver restart

2015-11-03 Thread vimal dinakaran
I have a simple spark streaming application which reads the data from the rabbitMQ and does some aggregation on window interval of 1 min and 1 hour for batch interval of 30s. I have a three node setup. And to enable checkpoint, I have mounted the same directory using sshfs to all worker node