On Wed, 22 Apr 2015 14:34:17 +0100 Bruce Richardson <bruce.richardson at intel.com> wrote:
> On Wed, Apr 22, 2015 at 07:36:59PM +0800, ChenXiaodong wrote: > > Thanks for you reply! There is actually only one NUMA node. here is what > > the commands return: > > > > Hi, > > there seems to be a mismatch in terms of the numa node the memory is reported > as being on (node 0) and the numa node that the cores are detected as being on > - as reported by the physical package id value. That's why the app is failing > as it's looking for the memory on the wrong numa node. > Can you try applying the patch I've just posted to change how we detect > the numa node of a core, and see if it helps in your case. > > thanks, > /Bruce This issue has been around a long time. Older kernels (pre 3.3) don't have a good solution and are really broken. Some old history: > On Fri, 31 May 2013 11:29:09 +0200 > Damien Millescamps <damien.millescamps at 6wind.com> wrote: > > > On 05/30/2013 07:12 PM, Stephen Hemminger wrote: > > > One fix not included is the NUMA cpu assignment. The original Intel code > > > in 1.2 incorrectly used /proc/cpuinfo to try and assign CPU's to NUMA > > > socket. > > > The problem is that /proc/cpuinfo physical_id corresponds to what the BIOS > > > tells the kernel and is intended for messages only. For example, on our > > > Dell boxes the first CPU and only CPU is reported as physical_id 1! > > > The fix is to use sysfs instead, > > Hi Stephen, > > > > Are you using the > > /sys/devices/system/node/nodeX/cpuX/topology/physical_package_id special > > file ? > > > > If so, then it is only usable starting from Kernel 3.3 according to this > > fix: > > http://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/commit/?id=64be4c1c2428e148de6081af235e2418e6a66dda > > > > The value returned for kernel prior to 3.3 is really not better than the > > one from /proc/cpuinfo, so that's kind of a robbing Peter to pay Paul > > example... > > > > There is obviously a problem with the NUMA node ID detection right now > > since both /proc/cpuinfo and /sys can return incorrect values, however > > the node and cpu numbering is always good in the kernel boot log from > > what I know. So there might be a better way to find the real node ID > > whatever the kernel version used. > > > > Cheers, > > You need to use /sys/devices/system/cpu/cpuN/topology/physical_package_id > as documented in kernel (Documentation/cputopology.txt). > This was confirmed by several kernel developers including Andi Kleen from > Intel. The value in /proc/cpuinfo comes from the APCI tables and is the value > reported by the BIOS. There are machines that report socket 1 and 2. > Haven't played with older kernels, but the topology information in sysfs > has existed since 2.6.16. > > The final Intel solution was to use physical_packate_id and fall back > to /proc/cpuinfo as last resort.