Re: Spark directStream with Kafka and process the lost messages.

Then,,, something is wrong in my code ;), thanks. 2015-11-30 16:46 GMT+01:00 Cody Koeninger : > Starting from the checkpoint using getOrCreate should be sufficient if all > you need is at-least-once semantics > > >

Spark directStream with Kafka and process the lost messages.

Hello, I have Spark and Kafka with directStream. I'm trying that if Spark dies it could process all those messages when it starts. The offsets are stored in chekpoints but I don't know how I could say to Spark to start in that point. I saw that there's another createDirectStream method with a

Re: Spark directStream with Kafka and process the lost messages.

Starting from the checkpoint using getOrCreate should be sufficient if all you need is at-least-once semantics http://spark.apache.org/docs/latest/streaming-programming-guide.html#checkpointing On Mon, Nov 30, 2015 at 9:38 AM, Guillermo Ortiz wrote: > Hello, > > I have