Package: lxc Version: 1:2.0.7-2+deb9u1 Severity: normal
Hi, I installed an unprivileged CentOS 7 container with $ lxc-create -n centos -t download after setting my system up according to the instructions given here: https://wiki.debian.org/LXC The resulting container starts as expected, but any attempt to shut it down again, fails. The container remains responsive, but just does not stop. Inside the container, the process list looks as follows: $ lxc-attach -n centos bash-4.2# ps auwwx USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.0 42544 2332 ? Ds Jan20 0:00 /sbin/init root 132 0.0 0.1 113380 13416 ? Ss Jan20 0:00 /sbin/dhclient -1 -q -lf /var/lib/dhclient/dhclient--eth0.lease -pf /var/run/dhclient-eth0.pid -H centos eth0 root 191 0.0 0.0 115396 3160 ? Ss Jan20 0:00 /bin/bash root 192 0.0 0.0 107912 616 ? D+ Jan20 0:00 sync root 193 0.0 0.0 115396 3064 ? Ss Jan20 0:00 /bin/bash root 194 0.0 0.0 151072 3564 ? R+ Jan20 0:00 ps auwwx bash-4.2# In a different shell, things look like this: $ lxc-attach -n centos bash-4.2# halt -n -f c^C^Z^C (ie, no reaction) In the kernel log of the host, I get a lot of these: Jan 19 22:20:37 debian kernel: [39269.678133] INFO: task systemd:6381 blocked for more than 120 seconds. Jan 19 22:20:37 debian kernel: [39269.678144] Tainted: G O 4.9.0-5-amd64 #1 Debian 4.9.65-3+deb9u2 Jan 19 22:20:37 debian kernel: [39269.678149] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Jan 19 22:20:37 debian kernel: [39269.678154] systemd D 0 6381 6371 0x00000104 Jan 19 22:20:37 debian kernel: [39269.678163] ffff976453d38000 ffff976553415800 ffff976504ffafc0 ffff97655fc98940 Jan 19 22:20:37 debian kernel: [39269.678170] ffff976500564480 ffffa7a7040b3bc0 ffffffffa6802923 ffff976504ffafc0 Jan 19 22:20:37 debian kernel: [39269.678176] 00ff976504ffafc0 ffff97655fc98940 ffff976504161dc0 ffff976504ffafc0 Jan 19 22:20:37 debian kernel: [39269.678181] Call Trace: Jan 19 22:20:37 debian kernel: [39269.678194] [<ffffffffa6802923>] ? __schedule+0x233/0x6d0 Jan 19 22:20:37 debian kernel: [39269.678200] [<ffffffffa6802df2>] ? schedule+0x32/0x80 Jan 19 22:20:37 debian kernel: [39269.678207] [<ffffffffa6805999>] ? rwsem_down_write_failed+0x1f9/0x360 Jan 19 22:20:37 debian kernel: [39269.678214] [<ffffffffa64801d0>] ? kernfs_sop_show_options+0x30/0x30 Jan 19 22:20:37 debian kernel: [39269.678220] [<ffffffffa6538213>] ? call_rwsem_down_write_failed+0x13/0x20 Jan 19 22:20:37 debian kernel: [39269.678225] [<ffffffffa6805039>] ? down_write+0x29/0x40 Jan 19 22:20:37 debian kernel: [39269.678231] [<ffffffffa64055ab>] ? grab_super+0x2b/0x90 Jan 19 22:20:37 debian kernel: [39269.678237] [<ffffffffa6405b43>] ? sget_userns+0x163/0x490 Jan 19 22:20:37 debian kernel: [39269.678242] [<ffffffffa6480240>] ? kernfs_sop_show_path+0x40/0x40 Jan 19 22:20:37 debian kernel: [39269.678246] [<ffffffffa648044a>] ? kernfs_mount_ns+0x7a/0x220 Jan 19 22:20:37 debian kernel: [39269.678252] [<ffffffffa630dba4>] ? cgroup_mount+0x334/0x810 Jan 19 22:20:37 debian kernel: [39269.678259] [<ffffffffa6406ae6>] ? mount_fs+0x36/0x150 Jan 19 22:20:37 debian kernel: [39269.678264] [<ffffffffa6423f32>] ? vfs_kern_mount+0x62/0x100 Jan 19 22:20:37 debian kernel: [39269.678268] [<ffffffffa64263ff>] ? do_mount+0x1cf/0xc80 Jan 19 22:20:37 debian kernel: [39269.678273] [<ffffffffa64271de>] ? SyS_mount+0x7e/0xd0 Jan 19 22:20:37 debian kernel: [39269.678279] [<ffffffffa6203b1c>] ? do_syscall_64+0x7c/0xf0 Jan 19 22:20:37 debian kernel: [39269.678285] [<ffffffffa68076ee>] ? entry_SYSCALL64_slow_path+0x25/0x25 Jan 19 22:20:37 debian kernel: [39269.678294] INFO: task sync:6958 blocked for more than 120 seconds. Jan 19 22:20:37 debian kernel: [39269.678300] Tainted: G O 4.9.0-5-amd64 #1 Debian 4.9.65-3+deb9u2 Jan 19 22:20:37 debian kernel: [39269.678303] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Jan 19 22:20:37 debian kernel: [39269.678307] sync D 0 6958 6951 0x00000104 Jan 19 22:20:37 debian kernel: [39269.678312] ffff976553fc4800 0000000000000000 ffff97653103a140 ffff97655fcd8940 Jan 19 22:20:37 debian kernel: [39269.678318] ffff976556666f00 ffffa7a706043df0 ffffffffa6802923 ffff9765500a5760 Jan 19 22:20:37 debian kernel: [39269.678324] 0000000000000286 ffff97655fcd8940 ffffa7a706043df8 ffff97653103a140 Jan 19 22:20:37 debian kernel: [39269.678329] Call Trace: Jan 19 22:20:37 debian kernel: [39269.678335] [<ffffffffa6802923>] ? __schedule+0x233/0x6d0 Jan 19 22:20:37 debian kernel: [39269.678340] [<ffffffffa6802df2>] ? schedule+0x32/0x80 Jan 19 22:20:37 debian kernel: [39269.678345] [<ffffffffa6805740>] ? rwsem_down_read_failed+0xf0/0x150 Jan 19 22:20:37 debian kernel: [39269.678350] [<ffffffffa641f86e>] ? iput+0x7e/0x210 Jan 19 22:20:37 debian kernel: [39269.678356] [<ffffffffa6436870>] ? SyS_tee+0x390/0x390 Jan 19 22:20:37 debian kernel: [39269.678361] [<ffffffffa65381e4>] ? call_rwsem_down_read_failed+0x14/0x30 Jan 19 22:20:37 debian kernel: [39269.678366] [<ffffffffa6804ffc>] ? down_read+0x1c/0x30 Jan 19 22:20:37 debian kernel: [39269.678371] [<ffffffffa64064ac>] ? iterate_supers+0x9c/0x100 Jan 19 22:20:37 debian kernel: [39269.678376] [<ffffffffa6436bc2>] ? sys_sync+0x42/0xb0 Jan 19 22:20:37 debian kernel: [39269.678380] [<ffffffffa6203b1c>] ? do_syscall_64+0x7c/0xf0 Jan 19 22:20:37 debian kernel: [39269.678386] [<ffffffffa68076ee>] ? entry_SYSCALL64_slow_path+0x25/0x25 Jan 19 22:20:37 debian kernel: [39269.678401] INFO: task halt:7352 blocked for more than 120 seconds. Jan 19 22:20:37 debian kernel: [39269.678406] Tainted: G O 4.9.0-5-amd64 #1 Debian 4.9.65-3+deb9u2 Jan 19 22:20:37 debian kernel: [39269.678409] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Jan 19 22:20:37 debian kernel: [39269.678412] halt D 0 7352 7211 0x00000100 Jan 19 22:20:37 debian kernel: [39269.678417] ffff976554b10000 0000000000000000 ffff97650de10200 ffff97655fc18940 Jan 19 22:20:37 debian kernel: [39269.678422] ffffffffa6e11500 ffffa7a7059bfdf0 ffffffffa6802923 ffff9765500a5760 Jan 19 22:20:37 debian kernel: [39269.678428] 0000000000000286 ffff97655fc18940 ffffa7a7059bfdf8 ffff97650de10200 Jan 19 22:20:37 debian kernel: [39269.678433] Call Trace: Jan 19 22:20:37 debian kernel: [39269.678438] [<ffffffffa6802923>] ? __schedule+0x233/0x6d0 Jan 19 22:20:37 debian kernel: [39269.678443] [<ffffffffa6802df2>] ? schedule+0x32/0x80 Jan 19 22:20:37 debian kernel: [39269.678448] [<ffffffffa6805740>] ? rwsem_down_read_failed+0xf0/0x150 Jan 19 22:20:37 debian kernel: [39269.678452] [<ffffffffa641f86e>] ? iput+0x7e/0x210 Jan 19 22:20:37 debian kernel: [39269.678458] [<ffffffffa6436870>] ? SyS_tee+0x390/0x390 Jan 19 22:20:37 debian kernel: [39269.678463] [<ffffffffa65381e4>] ? call_rwsem_down_read_failed+0x14/0x30 Jan 19 22:20:37 debian kernel: [39269.678468] [<ffffffffa6804ffc>] ? down_read+0x1c/0x30 Jan 19 22:20:37 debian kernel: [39269.678473] [<ffffffffa64064ac>] ? iterate_supers+0x9c/0x100 Jan 19 22:20:37 debian kernel: [39269.678478] [<ffffffffa6436bc2>] ? sys_sync+0x42/0xb0 Jan 19 22:20:37 debian kernel: [39269.678482] [<ffffffffa6203b1c>] ? do_syscall_64+0x7c/0xf0 Jan 19 22:20:37 debian kernel: [39269.678487] [<ffffffffa68076ee>] ? entry_SYSCALL64_slow_path+0x25/0x25 Jan 19 22:22:38 debian kernel: [39390.499861] INFO: task systemd:6381 blocked for more than 120 seconds. Jan 19 22:22:38 debian kernel: [39390.499872] Tainted: G O 4.9.0-5-amd64 #1 Debian 4.9.65-3+deb9u2 Jan 19 22:22:38 debian kernel: [39390.499877] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Jan 19 22:22:38 debian kernel: [39390.499881] systemd D 0 6381 6371 0x00000104 Jan 19 22:22:38 debian kernel: [39390.499890] ffff976453d38000 ffff976553415800 ffff976504ffafc0 ffff97655fc98940 Jan 19 22:22:38 debian kernel: [39390.499897] ffff976500564480 ffffa7a7040b3bc0 ffffffffa6802923 ffff976504ffafc0 Jan 19 22:22:38 debian kernel: [39390.499903] 00ff976504ffafc0 ffff97655fc98940 ffff976504161dc0 ffff976504ffafc0 Jan 19 22:22:38 debian kernel: [39390.499908] Call Trace: Jan 19 22:22:38 debian kernel: [39390.499922] [<ffffffffa6802923>] ? __schedule+0x233/0x6d0 Jan 19 22:22:38 debian kernel: [39390.499927] [<ffffffffa6802df2>] ? schedule+0x32/0x80 Jan 19 22:22:38 debian kernel: [39390.499934] [<ffffffffa6805999>] ? rwsem_down_write_failed+0x1f9/0x360 Jan 19 22:22:38 debian kernel: [39390.499941] [<ffffffffa64801d0>] ? kernfs_sop_show_options+0x30/0x30 Jan 19 22:22:38 debian kernel: [39390.499947] [<ffffffffa6538213>] ? call_rwsem_down_write_failed+0x13/0x20 Jan 19 22:22:38 debian kernel: [39390.499952] [<ffffffffa6805039>] ? down_write+0x29/0x40 Jan 19 22:22:38 debian kernel: [39390.499958] [<ffffffffa64055ab>] ? grab_super+0x2b/0x90 Jan 19 22:22:38 debian kernel: [39390.499963] [<ffffffffa6405b43>] ? sget_userns+0x163/0x490 Jan 19 22:22:38 debian kernel: [39390.499968] [<ffffffffa6480240>] ? kernfs_sop_show_path+0x40/0x40 Jan 19 22:22:38 debian kernel: [39390.499973] [<ffffffffa648044a>] ? kernfs_mount_ns+0x7a/0x220 Jan 19 22:22:38 debian kernel: [39390.499978] [<ffffffffa630dba4>] ? cgroup_mount+0x334/0x810 Jan 19 22:22:38 debian kernel: [39390.499985] [<ffffffffa6406ae6>] ? mount_fs+0x36/0x150 Jan 19 22:22:38 debian kernel: [39390.499990] [<ffffffffa6423f32>] ? vfs_kern_mount+0x62/0x100 Jan 19 22:22:38 debian kernel: [39390.499995] [<ffffffffa64263ff>] ? do_mount+0x1cf/0xc80 Jan 19 22:22:38 debian kernel: [39390.500000] [<ffffffffa64271de>] ? SyS_mount+0x7e/0xd0 Jan 19 22:22:38 debian kernel: [39390.500006] [<ffffffffa6203b1c>] ? do_syscall_64+0x7c/0xf0 Jan 19 22:22:38 debian kernel: [39390.500012] [<ffffffffa68076ee>] ? entry_SYSCALL64_slow_path+0x25/0x25 Jan 19 22:22:38 debian kernel: [39390.500021] INFO: task sync:6958 blocked for more than 120 seconds. Jan 19 22:22:38 debian kernel: [39390.500026] Tainted: G O 4.9.0-5-amd64 #1 Debian 4.9.65-3+deb9u2 Jan 19 22:22:38 debian kernel: [39390.500030] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Jan 19 22:22:38 debian kernel: [39390.500033] sync D 0 6958 6951 0x00000104 Jan 19 22:22:38 debian kernel: [39390.500038] ffff976553fc4800 0000000000000000 ffff97653103a140 ffff97655fcd8940 Jan 19 22:22:38 debian kernel: [39390.500045] ffff976556666f00 ffffa7a706043df0 ffffffffa6802923 ffff9765500a5760 Jan 19 22:22:38 debian kernel: [39390.500050] 0000000000000286 ffff97655fcd8940 ffffa7a706043df8 ffff97653103a140 Jan 19 22:22:38 debian kernel: [39390.500055] Call Trace: Jan 19 22:22:38 debian kernel: [39390.500061] [<ffffffffa6802923>] ? __schedule+0x233/0x6d0 Jan 19 22:22:38 debian kernel: [39390.500066] [<ffffffffa6802df2>] ? schedule+0x32/0x80 Jan 19 22:22:38 debian kernel: [39390.500071] [<ffffffffa6805740>] ? rwsem_down_read_failed+0xf0/0x150 Jan 19 22:22:38 debian kernel: [39390.500076] [<ffffffffa641f86e>] ? iput+0x7e/0x210 Jan 19 22:22:38 debian kernel: [39390.500082] [<ffffffffa6436870>] ? SyS_tee+0x390/0x390 Jan 19 22:22:38 debian kernel: [39390.500088] [<ffffffffa65381e4>] ? call_rwsem_down_read_failed+0x14/0x30 Jan 19 22:22:38 debian kernel: [39390.500092] [<ffffffffa6804ffc>] ? down_read+0x1c/0x30 Jan 19 22:22:38 debian kernel: [39390.500098] [<ffffffffa64064ac>] ? iterate_supers+0x9c/0x100 Jan 19 22:22:38 debian kernel: [39390.500103] [<ffffffffa6436bc2>] ? sys_sync+0x42/0xb0 Jan 19 22:22:38 debian kernel: [39390.500107] [<ffffffffa6203b1c>] ? do_syscall_64+0x7c/0xf0 Jan 19 22:22:38 debian kernel: [39390.500112] [<ffffffffa68076ee>] ? entry_SYSCALL64_slow_path+0x25/0x25 Jan 19 22:22:38 debian kernel: [39390.500127] INFO: task halt:7352 blocked for more than 120 seconds. Jan 19 22:22:38 debian kernel: [39390.500132] Tainted: G O 4.9.0-5-amd64 #1 Debian 4.9.65-3+deb9u2 Jan 19 22:22:38 debian kernel: [39390.500135] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Jan 19 22:22:38 debian kernel: [39390.500138] halt D 0 7352 7211 0x00000100 Jan 19 22:22:38 debian kernel: [39390.500143] ffff976554b10000 0000000000000000 ffff97650de10200 ffff97655fc18940 Jan 19 22:22:38 debian kernel: [39390.500149] ffffffffa6e11500 ffffa7a7059bfdf0 ffffffffa6802923 ffff9765500a5760 Jan 19 22:22:38 debian kernel: [39390.500154] 0000000000000286 ffff97655fc18940 ffffa7a7059bfdf8 ffff97650de10200 Jan 19 22:22:38 debian kernel: [39390.500159] Call Trace: Jan 19 22:22:38 debian kernel: [39390.500165] [<ffffffffa6802923>] ? __schedule+0x233/0x6d0 Jan 19 22:22:38 debian kernel: [39390.500169] [<ffffffffa6802df2>] ? schedule+0x32/0x80 Jan 19 22:22:38 debian kernel: [39390.500174] [<ffffffffa6805740>] ? rwsem_down_read_failed+0xf0/0x150 Jan 19 22:22:38 debian kernel: [39390.500179] [<ffffffffa641f86e>] ? iput+0x7e/0x210 Jan 19 22:22:38 debian kernel: [39390.500184] [<ffffffffa6436870>] ? SyS_tee+0x390/0x390 Jan 19 22:22:38 debian kernel: [39390.500189] [<ffffffffa65381e4>] ? call_rwsem_down_read_failed+0x14/0x30 Jan 19 22:22:38 debian kernel: [39390.500194] [<ffffffffa6804ffc>] ? down_read+0x1c/0x30 Jan 19 22:22:38 debian kernel: [39390.500199] [<ffffffffa64064ac>] ? iterate_supers+0x9c/0x100 Jan 19 22:22:38 debian kernel: [39390.500204] [<ffffffffa6436bc2>] ? sys_sync+0x42/0xb0 Jan 19 22:22:38 debian kernel: [39390.500208] [<ffffffffa6203b1c>] ? do_syscall_64+0x7c/0xf0 Jan 19 22:22:38 debian kernel: [39390.500213] [<ffffffffa68076ee>] ? entry_SYSCALL64_slow_path+0x25/0x25 Trying to 'lxc-stop -n centos' (name of the container) also does not work. A KVM machine with CentOS 7 on the same host works without problems. From the host machine, the container's processes look like this: $ ps auwwx|grep 231072 231072 6381 0.0 0.0 42544 2100 ? Ds 22:10 0:00 /sbin/init 231072 6708 0.0 0.1 113380 14640 ? Ss 22:12 0:00 /sbin/dhclient -1 -q -lf /var/lib/dhclient/dhclient--eth0.lease -pf /var/run/dhclient-eth0.pid -H centos eth0 231072 6951 0.0 0.0 115396 3160 pts/17 Ss 22:15 0:00 /bin/bash 231072 6958 0.0 0.0 107912 616 pts/17 D+ 22:15 0:00 sync 231072 7211 0.0 0.0 115396 3168 pts/18 Ss 22:16 0:00 /bin/bash 231072 7352 0.0 0.0 128432 2576 pts/18 D+ 22:18 0:00 halt -f 231072 7525 0.0 0.0 115396 3116 pts/20 Ss 22:19 0:00 /bin/bash 231072 7589 0.0 0.0 128432 2796 pts/20 D+ 22:19 0:00 halt -n -f toni 10162 0.0 0.0 12720 980 pts/5 S+ 22:49 0:00 grep --color=auto 231072 231072 27515 0.0 0.0 1508 8 ? Ss 18:55 0:00 /sbin/init 231072 27827 0.0 0.0 1508 48 ? Ss 18:55 0:00 /sbin/syslogd -Z 231072 27848 0.0 0.0 1512 48 ? Ss 18:55 0:00 /usr/sbin/crond -c /etc/crontabs 231072 27917 0.0 0.0 1508 48 ? Ss 18:56 0:00 udhcpc -b -p /var/run/udhcpc.eth0.pid -i eth0 -x hostname:alpine 231072 27944 0.0 0.0 7320 308 ? S 18:56 0:00 /usr/sbin/sshd 231072 27948 0.0 0.0 1508 4 pts/0 Ss+ 18:56 0:00 /sbin/getty 38400 tty1 231072 27949 0.0 0.0 1508 4 pts/1 Ss+ 18:56 0:00 /sbin/getty 38400 tty2 231072 27950 0.0 0.0 1508 4 pts/2 Ss+ 18:56 0:00 /sbin/getty 38400 tty3 231072 27951 0.0 0.0 1508 4 pts/3 Ss+ 18:56 0:00 /sbin/getty 38400 tty4 231072 27952 0.0 0.0 1508 4 pts/9 Ss+ 18:56 0:00 /sbin/getty 38400 console It would be great if lxc could prevent a container misbehaving like that. There's a side effect on the host machine: Since the 'init' process of the container hangs, the host machine cannot shut down properly anymore. Cheers, Toni -- System Information: Debian Release: 9.3 APT prefers stable APT policy: (990, 'stable'), (90, 'testing'), (70, 'unstable') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 4.9.0-5-amd64 (SMP w/4 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.utf8 (charmap=UTF-8), LANGUAGE=en_US:en (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages lxc depends on: ii init-system-helpers 1.48 ii libapparmor1 2.11.0-3 ii libc6 2.24-11+deb9u1 ii libcap2 1:2.25-1 ii libgnutls30 3.5.8-5+deb9u3 ii liblxc1 1:2.0.7-2+deb9u1 ii libseccomp2 2.3.1-2.1 ii libselinux1 2.6-3+b3 ii lsb-base 9.20161125 ii python3 3.5.3-1 ii python3-lxc 1:2.0.7-2+deb9u1 Versions of packages lxc recommends: ii bridge-utils 1.5-13+deb9u1 ii debootstrap 1.0.89 ii dirmngr 2.1.18-8~deb9u1 ii dnsmasq-base 2.76-5+deb9u1 ii gnupg 2.1.18-8~deb9u1 ii iptables 1.6.0+snapshot20161117-6 ii libpam-cgfs 2.0.7-1 ii lxcfs 2.0.7-1 ii openssl 1.1.0f-3+deb9u1 ii rsync 3.1.2-1+deb9u1 ii uidmap 1:4.4-4.1 Versions of packages lxc suggests: pn apparmor <none> ii btrfs-tools 4.7.3-1 ii lvm2 2.02.168-2 -- Configuration Files: /etc/lxc/default.conf changed: lxc.network.type = veth lxc.network.link = virbr0 lxc.network.flags = up lxc.network.hwaddr = 00:16:3e:xx:xx:xx -- no debconf information