Then,,, something is wrong in my code ;), thanks. 2015-11-30 16:46 GMT+01:00 Cody Koeninger <c...@koeninger.org>:
> Starting from the checkpoint using getOrCreate should be sufficient if all > you need is at-least-once semantics > > > http://spark.apache.org/docs/latest/streaming-programming-guide.html#checkpointing > > On Mon, Nov 30, 2015 at 9:38 AM, Guillermo Ortiz <konstt2...@gmail.com> > wrote: > >> Hello, >> >> I have Spark and Kafka with directStream. I'm trying that if Spark dies >> it could process all those messages when it starts. The offsets are stored >> in chekpoints but I don't know how I could say to Spark to start in that >> point. >> I saw that there's another createDirectStream method with a fromOffsets >> parameter but, how could I access to the offsets? >> >> val ssc = new StreamingContext(sparkConf, Seconds(5)) >> ssc.checkpoint(checkpoint) >> val directKafkaStream = KafkaUtils.createDirectStream[String, String, >> StringDecoder, StringDecoder](ssc, kafkaBrokers, topic) >> >> >
