Kill the job in the middle of a batch, look at the worker logs to see which offsets were being processed, verify the messages for those offsets are read when you start the job back up
On Thu, Aug 27, 2015 at 10:14 AM, Hafiz Mujadid <hafizmujadi...@gmail.com> wrote: > Hi! > > I have enables check pointing in spark streaming with kafka. I can see that > spark streaming is checkpointing to the mentioned directory at hdfs. How > can > i test that it works fine and recover with no data loss ? > > > Thanks > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Writing-test-case-for-spark-streaming-checkpointing-tp24475.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
