Hi, With nonstreaming spark application, did checkpoint the RDD and I could see the RDD getting checkpointed. I have killed the application after checkpointing the RDD and restarted the same application again immediately, but it doesn't seem to pick from checkpoint and it again checkpoints the RDD. Could anyone please explain why am I seeing this behavior, why it is not picking from the checkpoint and proceeding further from there on the second run of the same application. Would really help me understand spark checkpoint work flow if I can get some clarity on the behavior. Please let me know if I am missing something.
[root@checkpointDir]# ls 9dd1acf0-bef8-4a4f-bf0e-f7624334abc5 a4f14f43-e7c3-4f64-a980-8483b42bb11d [root@9dd1acf0-bef8-4a4f-bf0e-f7624334abc5]# ls -la total 0 drwxr-xr-x. 3 root root 20 May 26 16:26 . drwxr-xr-x. 4 root root 94 May 26 16:24 .. drwxr-xr-x. 2 root root 133 May 26 16:26 rdd-28 [root@priya-vm 9dd1acf0-bef8-4a4f-bf0e-f7624334abc5]# cd rdd-28/ [root@priya-vm rdd-28]# ls part-00000 part-00001 _partitioner Thanks -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-checkpoint-nonstreaming-tp28712.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
