Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/1002#issuecomment-45889538 Yeah the Securitymanager is adding both to handle the case the containers run as one user but is accessing HDFS as whoever specified in SPARK_USER, it also came up in the review. @vanzin can you tell me your yarn setup where it was showing yarn as the user in web UI? I have a single node setup with daemons running as yarn and then launch job as myself and it shows up as myself (without your change). Are you running your client as super user and then specifying SPARK_USER to access hdfs files?
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---