Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/204#issuecomment-39449932 Its not readable by other users in the intermediate directory. Sorry I didn't explain fully. That directory itself is 1777, sticky bit set, world writable. Each user creates a subdirectory in there that is only readable by them and superuser group, history files are then written to that user directory with permissions so that only that user and group can read/write. The reasons for it moving it to the done directory are more for management reasons. It orders them by date, and then within a date makes sure one directory doesn't get to big, deletes them after a retention period (having them ordered by date makes that simple), etc.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---