I've been working on some changes to how aurora submits tasks to mesos, specifically around Docker tasks, but I'd also like to see how people feel about making it more general.
Currently, when Aurora submits a task to mesos, it does NOT set command.user on the ExecutorInfo, this means that mesos configures the sandbox (mesos sandbox that is) as root, and launches the executor (thermos_executor in our case) as root as well. What then happens is that the executor then chown()s the sandbox it creates to the aurora role/user, and also setuid()s the runners it forks to that role/user. However, the executor itself is still running as root. My proposal / change is to set command.user to the aurora role by default, which will cause the executor to run as that user. I've tested this already, and no changes are needed to the executor, it will still try to chown the sandbox (which is fine since it already owns it), and setuid() the runners it forks (again, fine, since they're already running as that user). *The controversial part of this* however is I'd like to enable this behavior BY DEFAULT, and allow disabling it (reverting to the current behavior now) via a flag to the scheduler. My reasoning here is two fold. 1) It's a more secure default, preventing a compromised executor from doing things it shouldn't, and 2) we already have a lot of "flag bloat", and flags are hard enough to discover as they are. However, I do believe this should be considered as a "breaking change", particularly because of finicky PEX extraction for the executor. I'd like to hear people's thoughts on this.