On Thu, Oct 25, 2018 at 11:00:58PM +0530, Srikar Dronamraju wrote: > > But it doesn't solve the problem. > > > > You can create multiple partitions with cpusets but still have an > > unbound task in the root cgroup. That would suffer the exact same > > problems. > > > > Thing is, load-balancing, of any kind, should respect sched_domains, and > > currently numa balancing barely looks at it. > > Agreed that we should have looked at sched_domains. However I still believe > we can't have task->cpus_allowed with a mix of isolcpus and non-isolcpus. > won't it lead to inconsistent behaviour?
We currently can; and it depends on what you call inconsistent. > > The proposed patch puts the minimal constraints on the numa balancer to > > respect sched_domains; but doesn't yet correctly deal with hotplug. > > I was also thinking about hotplug. Also your proposed patch and even my > proposed patch don't seem to work well with the below scenario. > > # cat /sys/devices/system/cpu/possible > 0-31 > # cat /sys/devices/system/cpu/isolated > 1,5,9,13 > # cat hist.sh > echo 0 > /proc/sys/kernel/numa_balancing > cd /sys/fs/cgroup/cpuset > mkdir -p student > cp cpuset.mems student/ > cd student > echo "0-31" > cpuset.cpus > echo $$ > cgroup.procs > echo "1-8" > cpuset.cpus > /home/srikar/work/ebizzy-0.3/ebizzy -S 1000 & > PID=$! > sleep 10 > pidstat -p $! -t |tail -n +3 |head -n 10 > pidstat -p $$ -t |tail -n +3 > pkill ebizzy > # > # ./hist.sh > 10:35:21 IST UID TGID TID %usr %system %guest %CPU CPU > Command > 10:35:21 IST 0 2645 - 8.70 0.01 0.00 8.71 1 > ebizzy > 10:35:21 IST 0 - 2645 0.01 0.00 0.00 0.01 1 > |__ebizzy > 10:35:21 IST 0 - 2647 0.14 0.00 0.00 0.14 1 > |__ebizzy > 10:35:21 IST 0 - 2648 0.13 0.00 0.00 0.13 1 > |__ebizzy > 10:35:21 IST 0 - 2649 0.13 0.00 0.00 0.13 1 > |__ebizzy > 10:35:21 IST 0 - 2650 0.13 0.00 0.00 0.13 1 > |__ebizzy > 10:35:21 IST 0 - 2651 0.13 0.00 0.00 0.13 1 > |__ebizzy > 10:35:21 IST 0 - 2652 0.13 0.00 0.00 0.13 1 > |__ebizzy > 10:35:21 IST 0 - 2653 0.13 0.00 0.00 0.13 1 > |__ebizzy > 10:35:23 IST UID TGID TID %usr %system %guest %CPU CPU > Command > 10:35:23 IST 0 2642 - 0.00 0.00 0.00 0.00 1 > hist.sh > 10:35:23 IST 0 - 2642 0.00 0.00 0.00 0.00 1 > |__hist.sh > # That's correct and specified behaviour. cpu1 has no sched domain (or the singleton domaon, which is the same) and thus its tasks will not be migrated. If there is a problem, it is with cpuset and its interaction with isolcpus. I still have to look at the last version of cpuset-v2, but iirc that was better in this regard. I'm stil saying we should kill isolcpus entirely, its rubbish, has always been.
