Hi, I've used hdfs 2.3.0-cdh5.0.1, mesos 0.19.1 and spark 1.0.2 that is re-compiled.
For a security reason, we run hdfs and mesos as hdfs, that is an account name and not in a root group, and non-root user submit a spark job on mesos. With no-switch_user, simple job, which only read data from hdfs, works fine, but when i code more complex job including map and groupBy, spark job always fails. When a spark job is running, it create temporary file into output dir on hdfs and the directory owner is hdfs - same account running mesos -. Finally, id to sumbit job fails to delete temporary file because of permission error. Please advise to solve this problem. JL
