@straber: I don't think this is a kernel bug after all. You said that LXD will not alter the CPU pinning in any way. But it must write _something_ to cpuset.cpus before you can add any tasks to the cgroup. So I assume that by "not altering" you mean it just copies the value from the parent cgroup. But by doing that you are including the isolated CPUs in the cpuset. And based on Documentation/kernel- parameters.txt, adding the isolated CPUs to the cpuset is a supported way of scheduling tasks to the isolated CPU.
If the kernel be default inherited the cpuset.cpus value from its parent it would be reasonable for it to keep the same behavior wrt isolated CPUs. But since you have to write a value I don't think there's any way the kernel can know whether or not you meant to include isolated CPUs. I think LXD will have to exclude them unless explicitly told not to. Note that the base cpuset must contain all possible CPUs, as any child cpuset must be a subset of its parent. The behavior I describe in #7 is still a bit weird though. I missed a point there, that cpuset.cpus had been written to and so the scheduler domains were rebuilt. I suspect it has to do with the fact that the load balancer won't migrate tasks to/from isolated CPUs. If the first task you added to the set was on a non-isolated CPU then it won't be migrated to an isolated CPU. And since (I think) any of its children will start on the same CPU as the parent process they wouldn't be migrated to an isolated CPU either. So in effect it still excludes the isolated CPU despite the scheduler domain being recalculated. One other thing does look a bit fishy. The base cgroup is special in that the cpus_allowed for tasks excludes the isolated cpus unless explicitly allowed via the CPU affinity syscalls. But if I move the task to a child cpuset then back to the base cpuset it sill has the isolated CPUs present in cpus_allowed. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1623143 Title: Linux container does not take same cpu configuration as kernet's hosts Status in linux package in Ubuntu: Confirmed Bug description: When I configured cpu affinity on a ubuntu 16.04 with kernel version "4.4.0-36-generic" on a host containing 3 cpus: on /etc/default/grub: GRUB_CMDLINE_LINUX_DEFAULT="isolcpus=0" update-grub reboot The output of "cat /proc/self/status" : ubuntu@ubuntu:~$ cat /proc/self/status . . . Cpus_allowed: 6 Cpus_allowed_list: 1-2 Mems_allowed: 00000000,00000001 Mems_allowed_list: 0 voluntary_ctxt_switches: 0 nonvoluntary_ctxt_switches: 1 When I create an lxd container in this host using these commands: lxc launch ubuntu:14.04 my-ubuntu And access to this container using: lxc exec my-ubuntu -- /bin/bash The output of "cat /proc/self/status" : . . . Cpus_allowed: 7 Cpus_allowed_list: 0-2 Mems_allowed: 00000000,00000001 Mems_allowed_list: 0 voluntary_ctxt_switches: 0 nonvoluntary_ctxt_switches: 4 Result: Container's process doesn't take same configuration as host's process. Host process use 2 cpus: Cpus_allowed_list: 1-2 Container process use 3 cpus: Cpus_allowed_list: 0-2 ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-36-generic 4.4.0-36.55 ProcVersionSignature: Ubuntu 4.4.0-36.55-generic 4.4.16 Uname: Linux 4.4.0-36-generic x86_64 AlsaVersion: Advanced Linux Sound Architecture Driver Version k4.4.0-36-generic. AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.1-0ubuntu2 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-path', '/dev/snd/hwC0D0', '/dev/snd/pcmC0D0c', '/dev/snd/pcmC0D0p', '/dev/snd/controlC0', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: Card0.Amixer.info: Error: [Errno 2] No such file or directory: 'amixer' Card0.Amixer.values: Error: [Errno 2] No such file or directory: 'amixer' Date: Tue Sep 13 18:36:29 2016 HibernationDevice: RESUME=/dev/mapper/ubuntu--vg-swap_1 InstallationDate: Installed on 2016-08-04 (40 days ago) InstallationMedia: Ubuntu-Server 16.04 LTS "Xenial Xerus" - Release amd64 (20160420.3) IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub MachineType: QEMU Standard PC (i440FX + PIIX, 1996) ProcEnviron: TERM=xterm-256color PATH=(custom, no user) XDG_RUNTIME_DIR=<set> LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: 0 qxldrmfb ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.4.0-36-generic root=/dev/mapper/username--vg-root ro isolcpus=0 RelatedPackageVersions: linux-restricted-modules-4.4.0-36-generic N/A linux-backports-modules-4.4.0-36-generic N/A linux-firmware 1.157 RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 04/01/2014 dmi.bios.vendor: SeaBIOS dmi.bios.version: Ubuntu-1.8.2-1ubuntu1 dmi.chassis.type: 1 dmi.chassis.vendor: QEMU dmi.chassis.version: pc-i440fx-wily dmi.modalias: dmi:bvnSeaBIOS:bvrUbuntu-1.8.2-1ubuntu1:bd04/01/2014:svnQEMU:pnStandardPC(i440FX+PIIX,1996):pvrpc-i440fx-wily:cvnQEMU:ct1:cvrpc-i440fx-wily: dmi.product.name: Standard PC (i440FX + PIIX, 1996) dmi.product.version: pc-i440fx-wily dmi.sys.vendor: QEMU To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1623143/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : [email protected] Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
