RDD saveAsObjectFile write to local file and HDFS

2014-11-26 Thread firemonk9
instead of hdfs when running on YARN cluster. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/RDD-saveAsObjectFile-write-to-local-file-and-HDFS-tp19898.html Sent from the Apache Spark User List mailing list archive at Nabble.com

Re: RDD saveAsObjectFile write to local file and HDFS

2014-11-26 Thread Daniel Haviv
saveAsObjectFile writes the file to hdfs. (ex : path /data/temp.txt ) Is there a way to explictly mention local file system instead of hdfs when running on YARN cluster. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/RDD-saveAsObjectFile-write-to-local-file

Re: RDD saveAsObjectFile write to local file and HDFS

2014-11-26 Thread Du Li
Add ³file://³ in front of your path. On 11/26/14, 10:15 AM, firemonk9 dhiraj.peech...@gmail.com wrote: When I am running spark locally, RDD saveAsObjectFile writes the file to local file system (ex : path /data/temp.txt) and when I am running spark on YARN cluster, RDD saveAsObjectFile