[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436542#comment-15436542 ] Hudson commented on YARN-5042: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10342 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10342/]) YARN-5042. Mount /sys/fs/cgroup into Docker containers as read only (vvasudev: rev 525d52bf7c18d1e334e02e6fb936400c9cf2b0bc) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/DockerRunCommand.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/TestDockerContainerRuntime.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/DockerLinuxContainerRuntime.java > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > Fix For: 2.9.0 > > Attachments: YARN-5042.001.patch, YARN-5042.002.patch, > YARN-5042.003.patch, YARN-5042.004.patch, YARN-5042.005.patch > > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15434225#comment-15434225 ] luhuichun commented on YARN-5042: - @Sidharta Seethana you mean create another JIRA to not create container work dir & log dir if they don't exist? > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > Attachments: YARN-5042.001.patch, YARN-5042.002.patch, > YARN-5042.003.patch, YARN-5042.004.patch, YARN-5042.005.patch > > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433544#comment-15433544 ] Sidharta Seethana commented on YARN-5042: - +1 for the latest patch from me as well. Though the existing default behavior is to create the mounts on the host system if they don't exist, I think it makes sense to not create container work dir, log dirs etc if they don't exist. That can be taken up in a separate JIRA. > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > Attachments: YARN-5042.001.patch, YARN-5042.002.patch, > YARN-5042.003.patch, YARN-5042.004.patch, YARN-5042.005.patch > > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433449#comment-15433449 ] Varun Vasudev commented on YARN-5042: - +1 for the latest patch. [~shaneku...@gmail.com], [~sidharta-s] - mind reviewing the latest patch? I'll commit it tomorrow unless someone objects. > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > Attachments: YARN-5042.001.patch, YARN-5042.002.patch, > YARN-5042.003.patch, YARN-5042.004.patch, YARN-5042.005.patch > > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425823#comment-15425823 ] luhuichun commented on YARN-5042: - @Varun Vasudev failed one unit test, but i have checked , it is nothing related to my patch, need review thx > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > Attachments: YARN-5042.001.patch, YARN-5042.002.patch, > YARN-5042.003.patch, YARN-5042.004.patch, YARN-5042.005.patch > > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423940#comment-15423940 ] Hadoop QA commented on YARN-5042: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 5s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 27s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 17s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 27s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 14s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 39s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 22s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 23s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 13s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 23s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 11s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 45s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 16s {color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 16s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 26m 23s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:9560f25 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12824073/YARN-5042.005.patch | | JIRA Issue | YARN-5042 | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 241dee1da519 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 2353271 | | Default Java | 1.8.0_101 | | findbugs | v3.0.0 | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/12801/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | unit test logs | https://builds.apache.org/job/PreCommit-YARN-Build/12801/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/12801/testReport/ | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/12801/console | | Powered by | Apache Yetus 0.3.0 http://yetus.apache.org | This message was automatically generated. > Mount
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15422291#comment-15422291 ] Varun Vasudev commented on YARN-5042: - Thanks for the updated patch [~luhuichun]! Just one change - {code} +.addMountLocation("/sys/fs/cgroup", "/sys/fs/cgroup:ro", true); {code} Can you please set createSource to false? We don't want to create /sys/fs/cgroup on the host machine if it doesn't exist. > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > Attachments: YARN-5042.001.patch, YARN-5042.002.patch, > YARN-5042.003.patch, YARN-5042.004.patch > > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416723#comment-15416723 ] Hadoop QA commented on YARN-5042: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 52s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 22s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 28s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 10m 20s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 15s {color} | {color:green} trunk passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s {color} | {color:blue} Skipped patched modules with no Java source: . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 56s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m 33s {color} | {color:green} trunk passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 15s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 50s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 50s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 26s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 9m 19s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 0s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s {color} | {color:blue} Skipped patched modules with no Java source: . {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 51s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 46s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 87m 7s {color} | {color:red} root in the patch failed. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 22s {color} | {color:red} The patch generated 3 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 155m 14s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency | | | hadoop.yarn.logaggregation.TestAggregatedLogFormat | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:9560f25 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12823174/YARN-5042.004.patch | | JIRA Issue | YARN-5042 | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 47766f219ba1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / a428d4f | | Default Java | 1.8.0_101 | | findbugs | v3.0.0 | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/12741/artifact/patchprocess/patch-unit-root.txt | | unit test logs | https://builds.apache.org/job/PreCommit-YARN-Build/12741/artifact/patchprocess/patch-unit-root.txt | | Test Results |
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408907#comment-15408907 ] luhuichun commented on YARN-5042: - @Varun Vasudev Hi Varun, thanks for your suggestions, you mean add a new function and modify the existing mountCommand to call this version? not modify the current addMountLocation? and also there is problem, if "/sys/fs/group" or other root privilege dirs does not exist, docker -v /etc/xxx will not run and raise an error > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > Attachments: YARN-5042.001.patch, YARN-5042.002.patch, > YARN-5042.003.patch > > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405606#comment-15405606 ] Varun Vasudev commented on YARN-5042: - [~luhuichun] - thanks for the latest patch. Some things need to be fixed - 1) {code} +if (new File("/sys/fs/cgroup").exists()) + runCommand.addMountLocation("/sys/fs/cgroup", "/sys/fs/cgroup:ro"); + {code} Instead of checking if the location exists in DockerLinuxContainerRuntime.java - can you add a new function in DockerRunCommand.java which takes the paths and a boolean as a param - which allows the user to toggle whether the mount should happen if the source doesn't exist. Something like - {code} public DockerRunCommand addMountLocation(String sourcePath, String destinationPath, boolean createSource) { boolean sourceExists = new File(sourcePath).exists(); if(!sourceExists && !createSource) { return this; } super.addCommandArguments("-v", sourcePath + ":" + destinationPath); return this; } {code} You should also modify the existing mountCommand to call this version with createSource as true because that's the current behavior. Adding the new function should also fix the FindBugs warning. 2) Please fix the failing unit test in TestDockerContainerRuntime.java. > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > Attachments: YARN-5042.001.patch, YARN-5042.002.patch, > YARN-5042.003.patch > > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405486#comment-15405486 ] Hadoop QA commented on YARN-5042: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 29s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 11s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 29s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 16s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 29s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 14s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 46s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 24s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 15s {color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 2 new + 18 unchanged - 0 fixed = 20 total (was 18) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 26s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 10s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s {color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 51s {color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 12s {color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 16s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 28m 5s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager | | | Hard coded reference to an absolute pathname in org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) At DockerLinuxContainerRuntime.java:absolute pathname in org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) At DockerLinuxContainerRuntime.java:[line 349] | | Failed junit tests | hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:9560f25 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12821784/YARN-5042.003.patch | | JIRA Issue | YARN-5042 | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 632d92a35025 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / d848184 | | Default Java | 1.8.0_101 | | findbugs | v3.0.0 | | checkstyle |
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15403039#comment-15403039 ] Sidharta Seethana commented on YARN-5042: - [~luhuichun], the latest jenkins run still shows broken unit tests - could you please update the tests? > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > Attachments: YARN-5042.001.patch, YARN-5042.002.patch > > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15402679#comment-15402679 ] Varun Vasudev commented on YARN-5042: - [~luhuichun] - can you please add a check to ensure /sys/fs/cgroup exists before adding it to the mount list? [~sidharta-s] - can you take a look at the patch and review it please? > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > Attachments: YARN-5042.001.patch, YARN-5042.002.patch > > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15401641#comment-15401641 ] Hadoop QA commented on YARN-5042: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 8s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 27s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 16s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 27s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 13s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 41s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 22s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 13s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 11s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 44s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 14s {color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 15s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 26m 26s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:9560f25 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12821274/YARN-5042.002.patch | | JIRA Issue | YARN-5042 | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 237ab3e173aa 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 34ccaa8 | | Default Java | 1.8.0_101 | | findbugs | v3.0.0 | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/12584/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | unit test logs | https://builds.apache.org/job/PreCommit-YARN-Build/12584/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/12584/testReport/ | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/12584/console | | Powered by | Apache Yetus 0.3.0 http://yetus.apache.org | This message was automatically generated. > Mount
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397498#comment-15397498 ] Zhankun Tang commented on YARN-5042: yes. It seems more and more linux distribution is going to adopt systemd as init system. > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > Attachments: YARN-5042.001.patch > > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397496#comment-15397496 ] Zhankun Tang commented on YARN-5042: I guess existing unit test should be changed to pass the unit test. > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > Attachments: YARN-5042.001.patch > > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397269#comment-15397269 ] Varun Vasudev commented on YARN-5042: - [~luhuichun] - can you please take a look at the test failures and fix them? Thanks! > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > Attachments: YARN-5042.001.patch > > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397268#comment-15397268 ] Varun Vasudev commented on YARN-5042: - [~tangzhankun] - the only reason to do this is to mount it by default - otherwise every user has to specify it every time. I agree that the generalize mount mechanism can do it as well. > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > Attachments: YARN-5042.001.patch > > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397241#comment-15397241 ] Hadoop QA commented on YARN-5042: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 41s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m 39s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 20s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 37s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 16s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 54s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 25s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 27s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 27s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 15s {color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 1 new + 5 unchanged - 0 fixed = 6 total (was 5) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 30s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 11s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 57s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 22s {color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 20s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 35m 44s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.nodemanager.containermanager.linux.runtime.TestDockerContainerRuntime | | | hadoop.yarn.server.nodemanager.TestDirectoryCollection | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:9560f25 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12820668/YARN-5042.001.patch | | JIRA Issue | YARN-5042 | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 1867fe1210c2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 8d06bda | | Default Java | 1.8.0_101 | | findbugs | v3.0.0 | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/12536/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/12536/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | unit test logs |
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393310#comment-15393310 ] Zhankun Tang commented on YARN-5042: I am not a systemd expert so one question is that why we need to mount /sys/fs/cgroup by default since application can mount it thru YARN-4595 already? Could you please explain a little more about the motivation on sytemd, [~vvasudev]? Thanks. > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15389080#comment-15389080 ] Varun Vasudev commented on YARN-5042: - [~luhuichun] - I've added you to the contributors list and assigned the ticket to you. Thanks! > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev >Assignee: luhuichun > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5042) Mount /sys/fs/cgroup into Docker containers as read only mount
[ https://issues.apache.org/jira/browse/YARN-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388793#comment-15388793 ] luhuichun commented on YARN-5042: - Hi, Varun Vasudev I am currently do this, can you assign this JIRA to me ? thx > Mount /sys/fs/cgroup into Docker containers as read only mount > -- > > Key: YARN-5042 > URL: https://issues.apache.org/jira/browse/YARN-5042 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Varun Vasudev > > Containers running systemd need access to /sys/fs/cgroup. We should mount it > into the container as a read only mount. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org