Well, it only uses my user name when I run my application in local mode
(i.e. spark is running on my laptop with a master url of "local".) Not
a general solution for you I'm afraid!
On 12/12/2013 5:38 PM, Koert Kuipers wrote:
Hey Philip,
how do you get spark to write to hdfs with your user name? When i use
spark it writes to hdfs as the user that runs the spark services... i
wish it read and wrote as me.
On Thu, Dec 12, 2013 at 6:37 PM, Philip Ogren <philip.og...@oracle.com
<mailto:philip.og...@oracle.com>> wrote:
When I call rdd.saveAsTextFile("hdfs://...") it uses my username
to write to the HDFS drive. If I try to write to an HDFS
directory that I do not have permissions to, then I get an error
like this:
Permission denied: user=me, access=WRITE,
inode="/user/you/":you:us:drwxr-xr-x
I can obviously get around this by changing the permissions on the
directory /user/you. However, is it possible to call
rdd.saveAsText with an alternate username and password?
Thanks,
Philip
