Re: Failed to save RDD as text file to local file system

I'm a little bit late but posting in case somebody googles this. It seems saveAsTextFile requires chmod 777 but the local directory won't default to give w to other users. I've tried saving to a mounted drive and was able to save without an error. Without the the "file", it won't save to the

Re: Failed to save RDD as text file to local file system

Hi, Could you just trying one thing. Make a directory any where out side cloudera and than try the same write. Suppose the directory made is testWrite. do r.saveAsTextFile(/home/testWrite/) I think cloudera/tmp folder do not have a write permission for users hosted other than the cloudera

RE: Failed to save RDD as text file to local file system

I want to save to local directory. I have tried the following and get error r.saveAsTextFile(file:/home/cloudera/tmp/out1) r.saveAsTextFile(file:///home/cloudera/tmp/out1) r.saveAsTextFile(file:home/cloudera/tmp/out1) They all generate the following error 15/01/12 08:31:10 WARN

RE: Failed to save RDD as text file to local file system

All right, I remove cloudera totally and install spark manually on bare Linux system and now r.saveAsTextFile(…) works. Thanks. Regards, Ningjun Wang Consulting Software Engineer LexisNexis 121 Chanlon Road New Providence, NJ 07974-1541 From: Prannoy [mailto:pran...@sigmoidanalytics.com]

Re: Failed to save RDD as text file to local file system

Have you tried simple giving the path where you want to save the file ? For instance in your case just do *r.saveAsTextFile(home/cloudera/tmp/out1) * Dont use* file* This will create a folder with name out1. saveAsTextFile always write by making a directory, it does not write data into a

Re: Failed to save RDD as text file to local file system

I think you're confusing HDFS paths and local paths. You are cd'ing to a directory and seem to want to write output there, but your path has no scheme and defaults to being an HDFS path. When you use file: you seem to have a permission error (perhaps). On Mon, Jan 12, 2015 at 4:21 PM, NingjunWang

RE: Failed to save RDD as text file to local file system

Prannoy I tried this r.saveAsTextFile(home/cloudera/tmp/out1), it return without error. But where does it saved to? The folder “/home/cloudera/tmp/out1” is not cretaed. I also tried the following cd /home/cloudera/tmp/ spark-shell scala val r = sc.parallelize(Array(a, b, c)) scala

Re: Failed to save RDD as text file to local file system

Have you found any resolution for this issue ? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Failed-to-save-RDD-as-text-file-to-local-file-system-tp21050p21067.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

RE: Failed to save RDD as text file to local file system

No, do you have any idea? Regards, Ningjun Wang Consulting Software Engineer LexisNexis 121 Chanlon Road New Providence, NJ 07974-1541 From: firemonk9 [via Apache Spark User List] [mailto:ml-node+s1001560n21067...@n3.nabble.com] Sent: Friday, January 09, 2015 2:56 PM To: Wang, Ningjun

Re: Failed to save RDD as text file to local file system

looks like it is trying to save the file in Hdfs. Check if you have set any hadoop path in your system. On Fri, Jan 9, 2015 at 12:14 PM, Raghavendra Pandey raghavendra.pan...@gmail.com wrote: Can you check permissions etc as I am able to run r.saveAsTextFile(file:///home/cloudera/tmp/out1)