[ https://issues.apache.org/jira/browse/YARN-8380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16551033#comment-16551033 ]
Billie Rinaldi edited comment on YARN-8380 at 7/20/18 5:46 PM: --------------------------------------------------------------- Ah, I see. I didn't realize some of the options were propagation options. I read over [https://docs.docker.com/storage/bind-mounts/] and it leads me to believe the following: * rw and ro are the "base" mode. If not specified, rw is assumed. * In addition to the "base" mode, a single bind propagation option can be specified (shared, rshared, slave, rslave, private, rprivate). * z and Z are options that modify the SELinux label of the host file being mounted into the container. It says to use extreme caution with these and sounds like we shouldn't allow them. * consistent, default, delegated, and cached are options that configure mount consistency in Mac OS X, so probably we don't need these. * nocopy is an option that controls whether data is copied from the container into the Docker volume on the host. It does not apply to bind mounts (files or directories not managed by Docker that already exist on the host) because copying is not performed for bind mounts. It seems like we are only supporting bind mounts at this point, but perhaps we should consider whether we would want to support Docker volumes in the future. Based on this analysis, the mount options the Docker runtime could parse from the user on the Java side are: # source:target:(rw|ro) # source:target:(rw|ro),(r?shared|r?slave|r?private) # source:target – here rw is assumed # source:target:(r?shared|r?slave|r?private) – here rw is assumed One question is whether we want to allow formats 3 and 4, or only allow 1 and 2 so the user has to specify rw explicitly (note that currently the user has to specify :ro or :rw). If we required format 1 or 2, to get a shared mount the user would specify source:target:rw,shared or source:target:ro,shared. Actually, since the mounts are passed as a comma-separated list, we'll need to use a different character to separate the propagation option. On the container-executor side, we can assume format 1 or 2 since it will have been converted on the Java side. It looks like I should parse the mount type to see if it has a bind propagation option or not. I am thinking that we should keep the original configs docker.allowed.rw-mounts and docker.allowed.ro-mounts and allow any of the bind propagation options (which I believe is what [~eyang] is suggesting). It seems like we might not need additional configs controlling whether the propagation options are allowed. What do you think? The propagation options are outlined here: [https://docs.docker.com/storage/bind-mounts/#configure-bind-propagation]. was (Author: billie.rinaldi): Ah, I see. I didn't realize some of the options were propagation options. I read over [https://docs.docker.com/storage/bind-mounts/] and it leads me to believe the following: * rw and ro are the "base" mode. If not specified, rw is assumed. * In addition to the "base" mode, a single bind propagation option can be specified (shared, rshared, slave, rslave, private, rprivate). * z and Z are options that modify the SELinux label of the host file being mounted into the container. It says to use extreme caution with these and sounds like we shouldn't allow them. * consistent, default, delegated, and cached are options that configure mount consistency in Mac OS X, so probably we don't need these. * nocopy is an option that controls whether data is copied from the container into the Docker volume on the host. It does not apply to bind mounts (files or directories not managed by Docker that already exist on the host) because copying is not performed for bind mounts. It seems like we are only supporting bind mounts at this point, but perhaps we should consider whether we would want to support Docker volumes in the future. Based on this analysis, the mount options the Docker runtime could parse from the user on the Java side are: # source:target:(rw|ro) # source:target:(rw|ro),(r?shared|r?slave|r?private) # source:target – here rw is assumed # source:target:(r?shared|r?slave|r?private) – here rw is assumed One question is whether we want to allow formats 3 and 4, or only allow 1 and 2 so the user has to specify rw explicitly (note that currently the user has to specify :ro or :rw). If we required format 1 or 2, to get a shared mount the user would specify source:target:rw,shared or source:target:ro,shared. On the container-executor side, we can assume format 1 or 2 since it will have been converted on the Java side. It looks like I should parse the mount type to see if it has a bind propagation option or not. I am thinking that we should keep the original configs docker.allowed.rw-mounts and docker.allowed.ro-mounts and allow any of the bind propagation options (which I believe is what [~eyang] is suggesting). It seems like we might not need additional configs controlling whether the propagation options are allowed. What do you think? The propagation options are outlined here: [https://docs.docker.com/storage/bind-mounts/#configure-bind-propagation]. > Support shared mounts in docker runtime > --------------------------------------- > > Key: YARN-8380 > URL: https://issues.apache.org/jira/browse/YARN-8380 > Project: Hadoop YARN > Issue Type: Bug > Reporter: Billie Rinaldi > Assignee: Billie Rinaldi > Priority: Major > Attachments: YARN-8380.1.patch, YARN-8380.2.patch > > > The docker run command supports the mount type shared, but currently we are > only supporting ro and rw mount types in the docker runtime. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org