Hi Koert, Spark with multi-user support has been merged in master branch with patch (https://github.com/apache/incubator-spark/pull/23), you can check out the master branch. These patch can support to access hdfs with the username you start the Spark application, not the one who starts Spark service.
Thanks Jerry From: Koert Kuipers [mailto:ko...@tresata.com] Sent: Friday, December 13, 2013 8:39 AM To: user@spark.incubator.apache.org Subject: Re: writing to HDFS with a given username Hey Philip, how do you get spark to write to hdfs with your user name? When i use spark it writes to hdfs as the user that runs the spark services... i wish it read and wrote as me. On Thu, Dec 12, 2013 at 6:37 PM, Philip Ogren <philip.og...@oracle.com<mailto:philip.og...@oracle.com>> wrote: When I call rdd.saveAsTextFile("hdfs://...") it uses my username to write to the HDFS drive. If I try to write to an HDFS directory that I do not have permissions to, then I get an error like this: Permission denied: user=me, access=WRITE, inode="/user/you/":you:us:drwxr-xr-x I can obviously get around this by changing the permissions on the directory /user/you. However, is it possible to call rdd.saveAsText with an alternate username and password? Thanks, Philip
