Thanks for the heads up, saw that and will make sure that is resolved before pulling into 0.9. Unless I'm missing something, they should just use sc.addJar to distributed the jar rather then relying on SPARK_YARN_APP_JAR.
Tom On Thursday, March 20, 2014 3:31 PM, Patrick Wendell <pwend...@gmail.com> wrote: Hey Tom, > I'll pull [SPARK-1053] Should not require SPARK_YARN_APP_JAR when running on > YARN - JIRA and [SPARK-1051] On Yarn, executors don't doAs as submitting > user - JIRA in. The pyspark one I would consider more of an enhancement so > might not be appropriate for a point release. Someone recently sent me a personal e-mail reporting some problems with this. I'll ask them to forward it to you/the dev list. Might be worth looking into before merging. > [SPARK-1051] On Yarn, executors don't doAs as submitting user - JIRA > This means that they can't write/read from files that the yarn user doesn't > have permissions to but the submitting user does. > View on spark-project.atlassian.net Preview by Yahoo Good call on this one. - Patrick
