[ https://issues.apache.org/jira/browse/MESOS-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15581706#comment-15581706 ]
Benjamin Bannier commented on MESOS-6357: ----------------------------------------- The test code looks innocent to me, and makes me wonder if this a true race in the {{mesos-containerizer}}. [~klueska] [~benjaminhindman] > `NestedMesosContainerizerTest.ROOT_CGROUPS_ParentExit` is flaky in Debian 8. > ---------------------------------------------------------------------------- > > Key: MESOS-6357 > URL: https://issues.apache.org/jira/browse/MESOS-6357 > Project: Mesos > Issue Type: Bug > Components: tests > Affects Versions: 1.1.0 > Environment: Debian 8 with SSL enabled > Reporter: Gilbert Song > Labels: flaky-test > > {noformat} > [00:21:51] : [Step 10/10] [ RUN ] > NestedMesosContainerizerTest.ROOT_CGROUPS_ParentExit > [00:21:51]W: [Step 10/10] I1008 00:21:51.357839 23530 > containerizer.cpp:202] Using isolation: > cgroups/cpu,filesystem/linux,namespaces/pid,network/cni,volume/image > [00:21:51]W: [Step 10/10] I1008 00:21:51.361143 23530 > linux_launcher.cpp:150] Using /sys/fs/cgroup/freezer as the freezer hierarchy > for the Linux launcher > [00:21:51]W: [Step 10/10] I1008 00:21:51.366930 23547 > containerizer.cpp:557] Recovering containerizer > [00:21:51]W: [Step 10/10] I1008 00:21:51.367962 23551 provisioner.cpp:253] > Provisioner recovery complete > [00:21:51]W: [Step 10/10] I1008 00:21:51.368253 23549 > containerizer.cpp:954] Starting container > 42589936-56b2-4e41-86d8-447bfaba4666 for executor 'executor' of framework > [00:21:51]W: [Step 10/10] I1008 00:21:51.368577 23548 cgroups.cpp:404] > Creating cgroup at > '/sys/fs/cgroup/cpu,cpuacct/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666' > for container 42589936-56b2-4e41-86d8-447bfaba4666 > [00:21:51]W: [Step 10/10] I1008 00:21:51.369863 23544 cpu.cpp:103] Updated > 'cpu.shares' to 1024 (cpus 1) for container > 42589936-56b2-4e41-86d8-447bfaba4666 > [00:21:51]W: [Step 10/10] I1008 00:21:51.370384 23545 > containerizer.cpp:1443] Launching 'mesos-containerizer' with flags > '--command="{"shell":true,"value":"read key <&30"}" --help="false" > --pipe_read="30" --pipe_write="34" > --pre_exec_commands="[{"arguments":["mesos-containerizer","mount","--help=false","--operation=make-rslave","--path=\/"],"shell":false,"value":"\/mnt\/teamcity\/work\/4240ba9ddd0997c3\/build\/src\/mesos-containerizer"},{"shell":true,"value":"mount > -n -t proc proc \/proc -o nosuid,noexec,nodev"}]" > --runtime_directory="/mnt/teamcity/temp/buildTmp/NestedMesosContainerizerTest_ROOT_CGROUPS_ParentExit_sEbtvQ/containers/42589936-56b2-4e41-86d8-447bfaba4666" > --unshare_namespace_mnt="false" > --working_directory="/mnt/teamcity/temp/buildTmp/NestedMesosContainerizerTest_ROOT_CGROUPS_ParentExit_MqjHi0"' > [00:21:51]W: [Step 10/10] I1008 00:21:51.370483 23544 > linux_launcher.cpp:421] Launching container > 42589936-56b2-4e41-86d8-447bfaba4666 and cloning with namespaces CLONE_NEWNS > | CLONE_NEWPID > [00:21:51]W: [Step 10/10] I1008 00:21:51.374867 23545 > containerizer.cpp:1480] Checkpointing container's forked pid 14139 to > '/mnt/teamcity/temp/buildTmp/NestedMesosContainerizerTest_ROOT_CGROUPS_ParentExit_gzjeKG/meta/slaves/frameworks/executors/executor/runs/42589936-56b2-4e41-86d8-447bfaba4666/pids/forked.pid' > [00:21:51]W: [Step 10/10] I1008 00:21:51.376519 23551 > containerizer.cpp:1648] Starting nested container > 42589936-56b2-4e41-86d8-447bfaba4666.a5bc9913-c32c-40c6-ab78-2b08910847f8 > [00:21:51]W: [Step 10/10] I1008 00:21:51.377296 23549 > containerizer.cpp:1443] Launching 'mesos-containerizer' with flags > '--command="{"shell":true,"value":"sleep 1000"}" --help="false" > --pipe_read="30" --pipe_write="34" > --pre_exec_commands="[{"arguments":["mesos-containerizer","mount","--help=false","--operation=make-rslave","--path=\/"],"shell":false,"value":"\/mnt\/teamcity\/work\/4240ba9ddd0997c3\/build\/src\/mesos-containerizer"},{"shell":true,"value":"mount > -n -t proc proc \/proc -o nosuid,noexec,nodev"}]" > --runtime_directory="/mnt/teamcity/temp/buildTmp/NestedMesosContainerizerTest_ROOT_CGROUPS_ParentExit_sEbtvQ/containers/42589936-56b2-4e41-86d8-447bfaba4666/containers/a5bc9913-c32c-40c6-ab78-2b08910847f8" > --unshare_namespace_mnt="false" > --working_directory="/mnt/teamcity/temp/buildTmp/NestedMesosContainerizerTest_ROOT_CGROUPS_ParentExit_MqjHi0/containers/a5bc9913-c32c-40c6-ab78-2b08910847f8"' > [00:21:51]W: [Step 10/10] I1008 00:21:51.377424 23548 > linux_launcher.cpp:421] Launching nested container > 42589936-56b2-4e41-86d8-447bfaba4666.a5bc9913-c32c-40c6-ab78-2b08910847f8 and > cloning with namespaces CLONE_NEWNS | CLONE_NEWPID > [00:21:51] : [Step 10/10] Executing pre-exec command > '{"arguments":["mesos-containerizer","mount","--help=false","--operation=make-rslave","--path=\/"],"shell":false,"value":"\/mnt\/teamcity\/work\/4240ba9ddd0997c3\/build\/src\/mesos-containerizer"}' > [00:21:51] : [Step 10/10] Executing pre-exec command > '{"shell":true,"value":"mount -n -t proc proc \/proc -o nosuid,noexec,nodev"}' > [00:21:51]W: [Step 10/10] sh: 1: Syntax error: Bad fd number > [00:21:51] : [Step 10/10] > ../../src/tests/containerizer/nested_mesos_containerizer_tests.cpp:451: > Failure > [00:21:51] : [Step 10/10] (launch).failure(): Failed to fork: Failed to > clone child process: Failed to synchronize child process > [00:21:51]W: [Step 10/10] I1008 00:21:51.506002 23544 cgroups.cpp:2705] > Freezing cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos/a5bc9913-c32c-40c6-ab78-2b08910847f8 > [00:21:51]W: [Step 10/10] I1008 00:21:51.507055 23546 cgroups.cpp:1439] > Successfully froze cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos/a5bc9913-c32c-40c6-ab78-2b08910847f8 > after 1.019904ms > [00:21:51]W: [Step 10/10] I1008 00:21:51.507967 23549 cgroups.cpp:2723] > Thawing cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos/a5bc9913-c32c-40c6-ab78-2b08910847f8 > [00:21:51]W: [Step 10/10] I1008 00:21:51.508783 23549 cgroups.cpp:1468] > Successfully thawed cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos/a5bc9913-c32c-40c6-ab78-2b08910847f8 > after 797952ns > [00:21:51]W: [Step 10/10] I1008 00:21:51.510030 23551 cgroups.cpp:2705] > Freezing cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos > [00:21:51]W: [Step 10/10] I1008 00:21:51.612181 23551 cgroups.cpp:1439] > Successfully froze cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos > after 102.128896ms > [00:21:51]W: [Step 10/10] I1008 00:21:51.613134 23548 cgroups.cpp:2723] > Thawing cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos > [00:21:51]W: [Step 10/10] I1008 00:21:51.614034 23547 cgroups.cpp:1468] > Successfully thawed cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos > after 878080ns > [00:21:51]W: [Step 10/10] I1008 00:21:51.615409 23550 cgroups.cpp:2705] > Freezing cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666 > [00:21:51]W: [Step 10/10] I1008 00:21:51.717521 23550 cgroups.cpp:1439] > Successfully froze cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666 > after 102.08512ms > [00:21:51]W: [Step 10/10] I1008 00:21:51.718426 23545 cgroups.cpp:2723] > Thawing cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666 > [00:21:51]W: [Step 10/10] I1008 00:21:51.719254 23547 cgroups.cpp:1468] > Successfully thawed cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666 > after 806912ns > [00:21:51]W: [Step 10/10] I1008 00:21:51.720582 23551 cgroups.cpp:2705] > Freezing cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d > [00:21:51]W: [Step 10/10] I1008 00:21:51.822980 23546 cgroups.cpp:1439] > Successfully froze cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d after > 102.373888ms > [00:21:51]W: [Step 10/10] I1008 00:21:51.824023 23544 cgroups.cpp:2723] > Thawing cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d > [00:21:51]W: [Step 10/10] I1008 00:21:51.824843 23546 cgroups.cpp:1468] > Successfully thawed cgroup > /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d after > 801024ns > [00:21:51] : [Step 10/10] [ FAILED ] > NestedMesosContainerizerTest.ROOT_CGROUPS_ParentExit (480 ms) > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)