i just call saveAsTextFile() twice. 'doc_topic_dist' is type of RDD[(Long,
Array[Int])],
each element is pair of (doc, topic_arr), for the same doc, they have
different of topic_arr in two files.
...
doc_topic_dist.coalesce(1, true).saveAsTextFile(save_path)
doc_topic_dist.coalesce(1, true).saveAsTextFile(save_path + "2")
...
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/two-calls-of-saveAsTextFile-have-different-results-on-the-same-RDD-tp4578.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
