[ https://issues.apache.org/jira/browse/YARN-9906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Brennan resolved YARN-9906. ------------------------------- Resolution: Invalid > When setting multi volumes throurh the "YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS" > setting is not valid > --------------------------------------------------------------------------------------------------- > > Key: YARN-9906 > URL: https://issues.apache.org/jira/browse/YARN-9906 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn > Reporter: lynn > Priority: Major > Attachments: docker_volume_mounts.patch > > > As > [https://hadoop.apache.org/docs/r3.1.0/hadoop-yarn/hadoop-yarn-site/DockerContainers.html#Application_Submission] > described, when I set the item "{{YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS" to > multi volumes mounts, the value is a comma-separated list of mounts.}} > > {quote}vars="YARN_CONTAINER_RUNTIME_TYPE=docker,YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=hadoop-docker, > > YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS=/etc/passwd:/etc/passwd:ro,/etc/group:/etc/group:ro;/etc/hadoop/conf:/etc/hadoop/conf" > hadoop jar hadoop-examples.jar pi -Dyarn.app.mapreduce.am.env=$vars \ > -Dmapreduce.map.env=$vars -Dmapreduce.reduce.env=$vars 10 100{quote} > I found the docker container can mount the first volume, so it can't be > running successfully without report error! > The code of > [DockerLinuxContainerRuntime.java|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/DockerLinuxContainerRuntime.java] > as follows: > {quote}if (environment.containsKey(ENV_DOCKER_CONTAINER_MOUNTS)) { > Matcher parsedMounts = USER_MOUNT_PATTERN.matcher( > environment.get(ENV_DOCKER_CONTAINER_MOUNTS)); > if (!parsedMounts.find()) { > throw new ContainerExecutionException( > "Unable to parse user supplied mount list: " > + environment.get(ENV_DOCKER_CONTAINER_MOUNTS)); > }{quote} > The regex pattern is in > [OCIContainerRuntime|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/OCIContainerRuntime.java] > as follows > {quote}static final Pattern USER_MOUNT_PATTERN = Pattern.compile( > "(?<=^|,)([^:\\x00]+):([^:\\x00]+)" + > "(:(r[ow]|(r[ow][+])?(r?shared|r?slave|r?private)))?(?:,|$)");{quote} > it is seperated by comma indeed, but when i read the code of submit the jar > to yarn , i find the code > [Apps.java|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Apps.java] > {quote}private static final Pattern VARVAL_SPLITTER = Pattern.compile( > "(?<=^|,)" // preceded by ',' or line begin > + '(' + Shell.ENV_NAME_REGEX + ')' // var group > + '=' > + "([^,]*)" // val group > ); > {quote} > It is sepearted by comma as the same. > So, I just modify the comma to semicolon(";") of the item > "YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS". -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org