[ https://issues.apache.org/jira/browse/YARN-9562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907493#comment-16907493 ]
Eric Badger commented on YARN-9562: ----------------------------------- {noformat:title=Required Configs} <property> <name>yarn.nodemanager.runtime.linux.allowed-runtimes</name> <value>docker,default,runc</value> </property> <property> <name>yarn.nodemanager.runtime.linux.runc.allowed-images</name> <value>$Name of image tag</value> </property> <property> <name>yarn.nodemanager.runtime.linux.runc.image-name</name> <value>$Name of image tag</value> </property> <property> <name>yarn.nodemanager.runtime.linux.runc.image-toplevel-dir</name> <value>/runc-root</value> </property> {noformat} {noformat:title=Doesn't need to be nscd, but you need some strategy to make sure that your username can be resolved in the container via its uid.} <property> <name>yarn.nodemanager.runtime.linux.runc.default-rw-mounts</name> <value>/var/run/nscd:/var/run/nscd</value> </property> {noformat} {noformat:title=At least 1 of the following 2 configs needs to be set. If you use the docker-to-squash tool from YARN 9564 then you should only need to set the hdfs hash file} <property> <name>yarn.nodemanager.runtime.linux.runc.image-tag-to-manifest-plugin.local-hash-file</name> <value>/home/ebadger/image-tag-to-hash</value> </property> <property> <name>yarn.nodemanager.runtime.linux.runc.image-tag-to-manifest-plugin.hdfs-hash-file</name> <value>/runc-root/image-tag-to-hash</value> </property> {noformat} [~eyang], I went ahead and moved the toplevel config into YarnConfiguration since it is relevant to multiple pieces of the runc runtime. So the config key is now {{yarn.nodemanager.runtime.linux.runc.image-toplevel-dir}}. I have copied and edited the original list of configs that I gave. Additionally, I have tested YARN-9562.004.patch with YARN-9561.003.patch and YARN-9564.003.patch together. {noformat} export vars="YARN_CONTAINER_RUNTIME_TYPE=runc,YARN_CONTAINER_RUNTIME_RUNC_IMAGE=hadoop/rhel6:current"; $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.*-tests.jar sleep -Dyarn.app.mapreduce.am.env="HADOOP_MAPRED_HOME=$HADOOP_HOME" -Dmapreduce.admin.user.env="HADOOP_MAPRED_HOME=$HADOOP_HOME" -Dyarn.app.mapreduce.am.env=$vars -Dmapreduce.map.env=$vars -Dmapreduce.reduce.env=$vars -mt 1 -rt 1 -m 1 -r 1 {noformat} I successfully ran the above sleep job (after running docker-to-squash.py on hadoop/rhel6:current). Also note that the image that you use will need to include java and any other Hadoop dependencies. We have created our own custom image that is based off of rhel6 that add java, zstandard, lzo, and some other packages. > Add Java changes for the new RuncContainerRuntime > ------------------------------------------------- > > Key: YARN-9562 > URL: https://issues.apache.org/jira/browse/YARN-9562 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Eric Badger > Assignee: Eric Badger > Priority: Major > Attachments: YARN-9562.001.patch, YARN-9562.002.patch, > YARN-9562.003.patch, YARN-9562.004.patch > > > This JIRA will be used to add the Java changes for the new > RuncContainerRuntime. This will work off of YARN-9560 to use much of the > existing DockerLinuxContainerRuntime code once it is moved up into an > abstract class that can be extended. -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org