[ https://issues.apache.org/jira/browse/YARN-7446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347865#comment-16347865 ]
Eric Yang commented on YARN-7446: --------------------------------- [~shaneku...@gmail.com] I understand that docker can run as user defined in the image or as someone else. The output generated by the user in the docker container will impact localized directory clean up. The described problem only exists in yarn mode (where we bind localized directory to docker container). We can solve the logging problem for yarn mode is to prevent multi-users container and disallow privileged container for yarn mode. This will align yarn-mode to the same design as YARN in Hadoop 2. The alternative is to tap into docker logs, and pipe (| tee /fileename) the stdout, stderr from the launch command to localize the output. Therefore the content is written to disk using end user credential instead of root user or other user that exists in the docker image. For docker mode (where we sandbox docker, and drop all mounts for untrusted image) and trusted image must reflect the uid/gid consistent to the host OS, hence writing to any remote volumes don't create security problems. We can call docker logs command to retrieve logs, which docker already buffer and manage properly. Docker rm command will delete the logs in the sandbox without privileges issue. This will not be an issue with log clean up. Let me know what you think about these approaches to solve the logging problem. Thanks > Docker container privileged mode and --user flag contradict each other > ---------------------------------------------------------------------- > > Key: YARN-7446 > URL: https://issues.apache.org/jira/browse/YARN-7446 > Project: Hadoop YARN > Issue Type: Sub-task > Affects Versions: 3.0.0 > Reporter: Eric Yang > Assignee: Eric Yang > Priority: Major > Attachments: YARN-7446.001.patch > > > In the current implementation, when privileged=true, --user flag is also > passed to docker for launching container. In reality, the container has no > way to use root privileges unless there is sticky bit or sudoers in the image > for the specified user to gain privileges again. To avoid duplication of > dropping and reacquire root privileges, we can reduce the duplication of > specifying both flag. When privileged mode is enabled, --user flag should be > omitted. When non-privileged mode is enabled, --user flag is supplied. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org