On 21/01/2021 22:17, Song Bao Hua (Barry Song) wrote: > > >> -----Original Message----- >> From: Dietmar Eggemann [mailto:dietmar.eggem...@arm.com] >> Sent: Friday, January 22, 2021 7:54 AM >> To: Valentin Schneider <valentin.schnei...@arm.com>; Meelis Roos >> <mr...@linux.ee>; LKML <linux-kernel@vger.kernel.org> >> Cc: Peter Zijlstra <pet...@infradead.org>; Vincent Guittot >> <vincent.guit...@linaro.org>; Song Bao Hua (Barry Song) >> <song.bao....@hisilicon.com>; Mel Gorman <mgor...@suse.de> >> Subject: Re: 5.11-rc4+git: Shortest NUMA path spans too many nodes >> >> On 21/01/2021 19:21, Valentin Schneider wrote: >>> On 21/01/21 19:39, Meelis Roos wrote:
[...] >> # cat /sys/devices/system/node/node*/distance >> 10 12 12 14 14 14 14 16 >> 12 10 14 12 14 14 12 14 >> 12 14 10 14 12 12 14 14 >> 14 12 14 10 12 12 14 14 >> 14 14 12 12 10 14 12 14 >> 14 14 12 12 14 10 14 12 >> 14 12 14 14 12 14 10 12 >> 16 14 14 14 14 12 12 10 >> >> The '16' seems to be the culprit. How does such a topo look like? Maybe like this: _________ | | .-6 0 4-. | \ / \ / | | 1 2 | | \ \ | --7 3----5 | | |____|_| |_______| > > Once we get a topology like this: > > > +------+ +------+ +-------+ +------+ > | node | |node | | node | |node | > | +---------+ +--------+ +-------+ | > +------+ +------+ +-------+ +------+ > > We can reproduce this issue. > For example, every cpu with the below numa_distance can have > "groups don't span domain->span": > node 0 1 2 3 > 0: 10 12 20 22 > 1: 12 10 22 24 > 2: 20 22 10 12 > 3: 22 24 12 10 2 20 2 So this should look like: 1 --- 0 ---- 2 --- 3