Hi,
I would like to share a RDD in several Spark Applications,
i.e, create one in application A, publish the ID somewhere and get the RDD back
directly using ID in Application B.
I know I can use Tachyon just as a filesystem and
s.saveAsTextFile("tachyon://localhost:19998/Y”) like this.
But get a RDD directly from tachyon instead of a file can sometimes avoid
parsing the same file repeatedly in different Apps, I think.
What am I supposed to do in order to share RDDs to get a better performance?
—
Best Regards!
Yijie Shen