I'm a little bit late but posting in case somebody googles this.
It seems saveAsTextFile requires chmod 777 but the local directory won't
default to give w to other users.
I've tried saving to a mounted drive and was able to save without an error.
Without the the "file", it won't save to the
Hi,
Could you just trying one thing. Make a directory any where out side
cloudera and than try the same write.
Suppose the directory made is testWrite.
do r.saveAsTextFile(/home/testWrite/)
I think cloudera/tmp folder do not have a write permission for users hosted
other than the cloudera
I want to save to local directory. I have tried the following and get error
r.saveAsTextFile(file:/home/cloudera/tmp/out1)
r.saveAsTextFile(file:///home/cloudera/tmp/out1)
r.saveAsTextFile(file:home/cloudera/tmp/out1)
They all generate the following error
15/01/12 08:31:10 WARN
All right, I remove cloudera totally and install spark manually on bare Linux
system and now r.saveAsTextFile(…) works.
Thanks.
Regards,
Ningjun Wang
Consulting Software Engineer
LexisNexis
121 Chanlon Road
New Providence, NJ 07974-1541
From: Prannoy [mailto:pran...@sigmoidanalytics.com]
Have you tried simple giving the path where you want to save the file ?
For instance in your case just do
*r.saveAsTextFile(home/cloudera/tmp/out1) *
Dont use* file*
This will create a folder with name out1. saveAsTextFile always write by
making a directory, it does not write data into a
I think you're confusing HDFS paths and local paths. You are cd'ing to
a directory and seem to want to write output there, but your path has
no scheme and defaults to being an HDFS path. When you use file: you
seem to have a permission error (perhaps).
On Mon, Jan 12, 2015 at 4:21 PM, NingjunWang
Prannoy
I tried this r.saveAsTextFile(home/cloudera/tmp/out1), it return without
error. But where does it saved to? The folder “/home/cloudera/tmp/out1” is not
cretaed.
I also tried the following
cd /home/cloudera/tmp/
spark-shell
scala val r = sc.parallelize(Array(a, b, c))
scala
Have you found any resolution for this issue ?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Failed-to-save-RDD-as-text-file-to-local-file-system-tp21050p21067.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
No, do you have any idea?
Regards,
Ningjun Wang
Consulting Software Engineer
LexisNexis
121 Chanlon Road
New Providence, NJ 07974-1541
From: firemonk9 [via Apache Spark User List]
[mailto:ml-node+s1001560n21067...@n3.nabble.com]
Sent: Friday, January 09, 2015 2:56 PM
To: Wang, Ningjun
looks like it is trying to save the file in Hdfs.
Check if you have set any hadoop path in your system.
On Fri, Jan 9, 2015 at 12:14 PM, Raghavendra Pandey
raghavendra.pan...@gmail.com wrote:
Can you check permissions etc as I am able to run
r.saveAsTextFile(file:///home/cloudera/tmp/out1)
10 matches
Mail list logo