I hate to be "that guy", but I'd like to know myself. I tried to setup something similar, except I created a "service" account which starts the Spark service, but like you, I kept on getting file permission errors when submitting jobs under my own login. My current workaround was to su to the service account and run under that login.
As ultimately what I am trying to achieve is an unattended, scheduled series of processes, to have a service account run everything is OK in my example. But at some stage, this behavior will always raise eyebrows from the security team where I work. I am sure a lot of this comes down to folder ownership and permissions and possibly ACL's. I haven't had the chance to deep dive myself, but hopefully bubbling this up might bring someone out of the woodwork with a solution. Cheers -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
