From: "Gautham R. Shenoy" <e...@linux.vnet.ibm.com> Hi,
This is the ninth iteration of the patchset to add support for big-core on POWER9. This patch also optimizes the task placement on such big-core systems. The previous versions can be found here: v8: https://lkml.org/lkml/2018/9/20/899 v7: https://lkml.org/lkml/2018/8/20/52 v6: https://lkml.org/lkml/2018/8/9/119 v5: https://lkml.org/lkml/2018/8/6/587 v4: https://lkml.org/lkml/2018/7/24/79 v3: https://lkml.org/lkml/2018/7/6/255 v2: https://lkml.org/lkml/2018/7/3/401 v1: https://lkml.org/lkml/2018/5/11/245 Changes : v8 --> v9: - Rebased it on v4.19-rc5 - Updated the commit log for the second patch as per Dave Hansen's suggestion. - Fixed the build errors reported by Michael Neuling and the Kernel Build bot. Description: ~~~~~~~~~~~~~~~~~~~~ IBM POWER9 SMT8 cores consists of two groups of small-cores where each group has its own L1 cache, translation cache and instruction-data flow. This can be discovered via the "ibm,thread-groups" CPU property in the device tree. Furthermore, on POWER9 the thread-ids of such a big-core is obtained by interleaving the thread-ids of the two small-cores. Eg: In an SMT8 core with thread ids {0,1,2,3,4,5,6,7}, the thread-ids of the threads in the two small-cores respectively will be {0,2,4,6} and {1,3,5,7} respectively. ------------------------- | L1 Cache | ---------------------------------- |L2| | | | | | | 0 | 2 | 4 | 6 |Small Core0 |C | | | | | Big |a -------------------------- Core |c | | | | | |h | 1 | 3 | 5 | 7 | Small Core1 |e | | | | | ----------------------------- | L1 Cache | -------------------------- On such a big-core system, when multiple tasks are scheduled to run on the big-core, we get the best performance when the tasks are spread across the pair of small-cores. Eg: Suppose there 4 tasks {p1, p2, p3, p4} are run on a big core, then An Example of Optimal Task placement: -------------------------- | | | | | | 0 | 2 | 4 | 6 | Small Core0 | (p1)| (p2)| | | Big Core -------------------------- | | | | | | 1 | 3 | 5 | 7 | Small Core1 | | (p3)| | (p4) | -------------------------- An example of Suboptimal Task placement: -------------------------- | | | | | | 0 | 2 | 4 | 6 | Small Core0 | (p1)| (p2)| | (p4)| Big Core -------------------------- | | | | | | 1 | 3 | 5 | 7 | Small Core1 | | (p3)| | | -------------------------- Currently on the big-core systems, the sched domain hierarchy is: SMT : group of CPUs in the SMT8 core. DIE : groups of CPUs on the same die. NUMA : all the CPUs in the system. Thus the scheduler doesn't distinguish between CPUs in the core that share the L1-cache vs the ones that don't resulting in a run-to-run variance when multithreaded applications are run on an SMT8 core. In this patch-set, we address this by defining the sched-domain on the big-core systems to be: SMT : group of CPUs sharing the L1 cache CACHE : group of CPUs in the SMT8 core. DIE : groups of CPUs on the same die. NUMA : all the CPUs in the system. With this, the Linux Kernel load-balancer will ensure that the tasks are spread across all the component small cores in the system, thereby yielding optimum performance. Furthermore, this solution works correctly across all SMT modes (8,4,2), as the interleaved thread-ids ensures that when we go to lower SMT modes (4,2) the threads are offlined in a descending order, thereby leaving equal number of threads from the component small cores online as illustrated below. This patchset contains three patches which on detecting the presence of big-cores, defines the SMT level sched domain to correspond to the threads of the small cores. Patch 1: adds support to detect the presence of big-cores and parses the output of "ibm,thread-groups" device-tree which using which it updates a per-cpu mask named cpu_smallcore_mask Patch 2: Defines the SMT level sched domain to correspond to the threads of the small cores. Patch 3: Creates a pair of sysfs attributes named /sys/devices/system/cpu/cpuN/topology/smallcore_thread_siblings and /sys/devices/system/cpu/cpuN/topology/smallcore_thread_siblings_list exposing the small-core siblings that share the L1 cache to the userspace. Results: ~~~~~~~~~~~~~~~~~ 1) 2 thread ebizzy ~~~~~~~~~~~~~~~~~~~~~~ Experimental results for ebizzy with 2 threads, bound to a single big-core show a marked improvement with this patchset over the 4.19.0-rc5 vanilla kernel. The result of 100 such runs for 4.19.0-rc5 kernel and the 4.19.0-rc5 + big-core-patches are as follows 4.19.0-rc5 vanilla ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ records/s : # samples : Histogram ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [0 - 1000000] : 0 : # [1000000 - 2000000] : 1 : # [2000000 - 3000000] : 8 : ## [3000000 - 4000000] : 31 : ####### [4000000 - 5000000] : 5 : ## [5000000 - 6000000] : 2 : # [6000000 - 7000000] : 53 : ########### ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 4.19.0-rc5 + big-core-patches ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ records/s : # samples : Histogram ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [0 - 1000000] : 0 : # [1000000 - 2000000] : 0 : # [2000000 - 3000000] : 7 : ## [3000000 - 4000000] : 8 : ## [4000000 - 5000000] : 0 : # [5000000 - 6000000] : 1 : # [6000000 - 7000000] : 84 : ################# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2) Hackbench (perf bench sched pipe) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 500 iterations of the hackbench run both on 4.19.0-rc5 vanilla kernel and v4.19.0-rc5 + big-core-patches. There isn't a significant difference between the two. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 4.19.0-rc5 vanilla ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ N Min Max Median Avg Stddev 500 5.195 7.108 6.818 6.634932 0.39956043 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 4.19.0-rc5 + big-core-patches ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ N Min Max Median Avg Stddev 500 5.215 6.978 6.895 6.759908 0.33699356 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Gautham R. Shenoy (3): powerpc: Detect the presence of big-cores via "ibm,thread-groups" powerpc: Use cpu_smallcore_sibling_mask at SMT level on bigcores powerpc/sysfs: Add topology/smallcore_thread_siblings[_list] Documentation/ABI/testing/sysfs-devices-system-cpu | 14 ++ arch/powerpc/include/asm/cputhreads.h | 2 + arch/powerpc/include/asm/smp.h | 6 + arch/powerpc/kernel/smp.c | 241 ++++++++++++++++++++- arch/powerpc/kernel/sysfs.c | 91 ++++++++ 5 files changed, 353 insertions(+), 1 deletion(-) -- 1.9.4