Kill the job in the middle of a batch, look at the worker logs to see which
offsets were being processed, verify the messages for those offsets are
read when you start the job back up
On Thu, Aug 27, 2015 at 10:14 AM, Hafiz Mujadid
wrote:
> Hi!
>
> I have enables check pointing in spark streaming with kafka. I can see that
> spark streaming is checkpointing to the mentioned directory at hdfs. How
> can
> i test that it works fine and recover with no data loss ?
>
>
> Thanks
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Writing-test-case-for-spark-streaming-checkpointing-tp24475.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>
