I have asked this question before but get no answer. Asking again. Can I save RDD to the local file system and then read it back on a spark cluster with multiple nodes?
rdd.saveAsObjectFile("file:///home/data/rdd1<file:///\\home\data\rdd1>") val rdd2 = sc.objectFile("file:///home/data/rdd1<file:///\\home\data\rdd1>") This will works if the cluster has only one node. But my cluster has 3 nodes and each node has a local dir called /home/data. Is rdd saved to the local dir across 3 nodes? If so, does sc.objectFile(...) smart enough to read the local dir in all 3 nodes to merge them into a single rdd? Ningjun