[
https://issues.apache.org/jira/browse/MESOS-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586542#comment-15586542
]
Kevin Klues commented on MESOS-6357:
------------------------------------
This is a bug in the test. Jie spent about 4 hours debugging this last month,
only to discover that {{sh}} doesn't like double digit fds following a {{<&}}.
[~gilbert] was supposed to be fix this test to force the use of {{bash}}
instead of {{sh}}. Did that fall through the cracks before committing?
> `NestedMesosContainerizerTest.ROOT_CGROUPS_ParentExit` is flaky in Debian 8.
> ----------------------------------------------------------------------------
>
> Key: MESOS-6357
> URL: https://issues.apache.org/jira/browse/MESOS-6357
> Project: Mesos
> Issue Type: Bug
> Components: tests
> Affects Versions: 1.1.0
> Environment: Debian 8 with SSL enabled
> Reporter: Gilbert Song
> Labels: flaky-test
>
> {noformat}
> [00:21:51] : [Step 10/10] [ RUN ]
> NestedMesosContainerizerTest.ROOT_CGROUPS_ParentExit
> [00:21:51]W: [Step 10/10] I1008 00:21:51.357839 23530
> containerizer.cpp:202] Using isolation:
> cgroups/cpu,filesystem/linux,namespaces/pid,network/cni,volume/image
> [00:21:51]W: [Step 10/10] I1008 00:21:51.361143 23530
> linux_launcher.cpp:150] Using /sys/fs/cgroup/freezer as the freezer hierarchy
> for the Linux launcher
> [00:21:51]W: [Step 10/10] I1008 00:21:51.366930 23547
> containerizer.cpp:557] Recovering containerizer
> [00:21:51]W: [Step 10/10] I1008 00:21:51.367962 23551 provisioner.cpp:253]
> Provisioner recovery complete
> [00:21:51]W: [Step 10/10] I1008 00:21:51.368253 23549
> containerizer.cpp:954] Starting container
> 42589936-56b2-4e41-86d8-447bfaba4666 for executor 'executor' of framework
> [00:21:51]W: [Step 10/10] I1008 00:21:51.368577 23548 cgroups.cpp:404]
> Creating cgroup at
> '/sys/fs/cgroup/cpu,cpuacct/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666'
> for container 42589936-56b2-4e41-86d8-447bfaba4666
> [00:21:51]W: [Step 10/10] I1008 00:21:51.369863 23544 cpu.cpp:103] Updated
> 'cpu.shares' to 1024 (cpus 1) for container
> 42589936-56b2-4e41-86d8-447bfaba4666
> [00:21:51]W: [Step 10/10] I1008 00:21:51.370384 23545
> containerizer.cpp:1443] Launching 'mesos-containerizer' with flags
> '--command="{"shell":true,"value":"read key <&30"}" --help="false"
> --pipe_read="30" --pipe_write="34"
> --pre_exec_commands="[{"arguments":["mesos-containerizer","mount","--help=false","--operation=make-rslave","--path=\/"],"shell":false,"value":"\/mnt\/teamcity\/work\/4240ba9ddd0997c3\/build\/src\/mesos-containerizer"},{"shell":true,"value":"mount
> -n -t proc proc \/proc -o nosuid,noexec,nodev"}]"
> --runtime_directory="/mnt/teamcity/temp/buildTmp/NestedMesosContainerizerTest_ROOT_CGROUPS_ParentExit_sEbtvQ/containers/42589936-56b2-4e41-86d8-447bfaba4666"
> --unshare_namespace_mnt="false"
> --working_directory="/mnt/teamcity/temp/buildTmp/NestedMesosContainerizerTest_ROOT_CGROUPS_ParentExit_MqjHi0"'
> [00:21:51]W: [Step 10/10] I1008 00:21:51.370483 23544
> linux_launcher.cpp:421] Launching container
> 42589936-56b2-4e41-86d8-447bfaba4666 and cloning with namespaces CLONE_NEWNS
> | CLONE_NEWPID
> [00:21:51]W: [Step 10/10] I1008 00:21:51.374867 23545
> containerizer.cpp:1480] Checkpointing container's forked pid 14139 to
> '/mnt/teamcity/temp/buildTmp/NestedMesosContainerizerTest_ROOT_CGROUPS_ParentExit_gzjeKG/meta/slaves/frameworks/executors/executor/runs/42589936-56b2-4e41-86d8-447bfaba4666/pids/forked.pid'
> [00:21:51]W: [Step 10/10] I1008 00:21:51.376519 23551
> containerizer.cpp:1648] Starting nested container
> 42589936-56b2-4e41-86d8-447bfaba4666.a5bc9913-c32c-40c6-ab78-2b08910847f8
> [00:21:51]W: [Step 10/10] I1008 00:21:51.377296 23549
> containerizer.cpp:1443] Launching 'mesos-containerizer' with flags
> '--command="{"shell":true,"value":"sleep 1000"}" --help="false"
> --pipe_read="30" --pipe_write="34"
> --pre_exec_commands="[{"arguments":["mesos-containerizer","mount","--help=false","--operation=make-rslave","--path=\/"],"shell":false,"value":"\/mnt\/teamcity\/work\/4240ba9ddd0997c3\/build\/src\/mesos-containerizer"},{"shell":true,"value":"mount
> -n -t proc proc \/proc -o nosuid,noexec,nodev"}]"
> --runtime_directory="/mnt/teamcity/temp/buildTmp/NestedMesosContainerizerTest_ROOT_CGROUPS_ParentExit_sEbtvQ/containers/42589936-56b2-4e41-86d8-447bfaba4666/containers/a5bc9913-c32c-40c6-ab78-2b08910847f8"
> --unshare_namespace_mnt="false"
> --working_directory="/mnt/teamcity/temp/buildTmp/NestedMesosContainerizerTest_ROOT_CGROUPS_ParentExit_MqjHi0/containers/a5bc9913-c32c-40c6-ab78-2b08910847f8"'
> [00:21:51]W: [Step 10/10] I1008 00:21:51.377424 23548
> linux_launcher.cpp:421] Launching nested container
> 42589936-56b2-4e41-86d8-447bfaba4666.a5bc9913-c32c-40c6-ab78-2b08910847f8 and
> cloning with namespaces CLONE_NEWNS | CLONE_NEWPID
> [00:21:51] : [Step 10/10] Executing pre-exec command
> '{"arguments":["mesos-containerizer","mount","--help=false","--operation=make-rslave","--path=\/"],"shell":false,"value":"\/mnt\/teamcity\/work\/4240ba9ddd0997c3\/build\/src\/mesos-containerizer"}'
> [00:21:51] : [Step 10/10] Executing pre-exec command
> '{"shell":true,"value":"mount -n -t proc proc \/proc -o nosuid,noexec,nodev"}'
> [00:21:51]W: [Step 10/10] sh: 1: Syntax error: Bad fd number
> [00:21:51] : [Step 10/10]
> ../../src/tests/containerizer/nested_mesos_containerizer_tests.cpp:451:
> Failure
> [00:21:51] : [Step 10/10] (launch).failure(): Failed to fork: Failed to
> clone child process: Failed to synchronize child process
> [00:21:51]W: [Step 10/10] I1008 00:21:51.506002 23544 cgroups.cpp:2705]
> Freezing cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos/a5bc9913-c32c-40c6-ab78-2b08910847f8
> [00:21:51]W: [Step 10/10] I1008 00:21:51.507055 23546 cgroups.cpp:1439]
> Successfully froze cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos/a5bc9913-c32c-40c6-ab78-2b08910847f8
> after 1.019904ms
> [00:21:51]W: [Step 10/10] I1008 00:21:51.507967 23549 cgroups.cpp:2723]
> Thawing cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos/a5bc9913-c32c-40c6-ab78-2b08910847f8
> [00:21:51]W: [Step 10/10] I1008 00:21:51.508783 23549 cgroups.cpp:1468]
> Successfully thawed cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos/a5bc9913-c32c-40c6-ab78-2b08910847f8
> after 797952ns
> [00:21:51]W: [Step 10/10] I1008 00:21:51.510030 23551 cgroups.cpp:2705]
> Freezing cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos
> [00:21:51]W: [Step 10/10] I1008 00:21:51.612181 23551 cgroups.cpp:1439]
> Successfully froze cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos
> after 102.128896ms
> [00:21:51]W: [Step 10/10] I1008 00:21:51.613134 23548 cgroups.cpp:2723]
> Thawing cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos
> [00:21:51]W: [Step 10/10] I1008 00:21:51.614034 23547 cgroups.cpp:1468]
> Successfully thawed cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666/mesos
> after 878080ns
> [00:21:51]W: [Step 10/10] I1008 00:21:51.615409 23550 cgroups.cpp:2705]
> Freezing cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666
> [00:21:51]W: [Step 10/10] I1008 00:21:51.717521 23550 cgroups.cpp:1439]
> Successfully froze cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666
> after 102.08512ms
> [00:21:51]W: [Step 10/10] I1008 00:21:51.718426 23545 cgroups.cpp:2723]
> Thawing cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666
> [00:21:51]W: [Step 10/10] I1008 00:21:51.719254 23547 cgroups.cpp:1468]
> Successfully thawed cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d/42589936-56b2-4e41-86d8-447bfaba4666
> after 806912ns
> [00:21:51]W: [Step 10/10] I1008 00:21:51.720582 23551 cgroups.cpp:2705]
> Freezing cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d
> [00:21:51]W: [Step 10/10] I1008 00:21:51.822980 23546 cgroups.cpp:1439]
> Successfully froze cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d after
> 102.373888ms
> [00:21:51]W: [Step 10/10] I1008 00:21:51.824023 23544 cgroups.cpp:2723]
> Thawing cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d
> [00:21:51]W: [Step 10/10] I1008 00:21:51.824843 23546 cgroups.cpp:1468]
> Successfully thawed cgroup
> /sys/fs/cgroup/freezer/mesos_test_458f8018-67e7-4cc6-8126-a535974db35d after
> 801024ns
> [00:21:51] : [Step 10/10] [ FAILED ]
> NestedMesosContainerizerTest.ROOT_CGROUPS_ParentExit (480 ms)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
