[AMD Official Use Only] Validated the changes on: - AMD EPYC 7713 - AMD EPYC 7543P - Intel(R) Xeon(R) Platinum 8352V
-----Original Message----- From: Varghese, Vipin <vipin.vargh...@amd.com> Sent: Monday, March 28, 2022 8:27 AM To: Thomas Monjalon <tho...@monjalon.net> Cc: david.march...@redhat.com; Tummala, Sivaprasad <sivaprasad.tumm...@amd.com>; dev@dpdk.org Subject: RE: [PATCH] usertools: enhance logic to display NUMA [AMD Official Use Only] Hi Thomas, <snipp> 26/03/2022 08:32, Vipin Varghese: > enhance python logic to accomadate NUMA information. Current logic > considers physical socket with CPU threads to core map. With new AMD > SKU vairant NUMA is no longer same as SOCKET. Single physical socket > can be partitioned to variant of 1,2 and 4. > > The changes address the new mapping with Socket-NUMA to CPU cores. > > Signed-off-by: Vipin Varghese <vipin.vargh...@amd.com> > --- > usertools/cpu_layout.py | 76 > +++++++++++++++++++++++++---------------- > 1 file changed, 47 insertions(+), 29 deletions(-) Honestly, I'm not sure it is a good idea to keep this script in the DPDK repo. Can it be replaced with hwloc usage? thanks for the suggestion, it is genuine and useful. Following is my observations - It takes some effort to identify the NUMA with `Group` - One needs to install ` lstopo-no-graphics` on distro and manually build and add on custom Linux. What is the output on the new AMD SKU for this command? lstopo-no-graphics --merge I have tried ` lstopo-no-graphics --merge` on a ` 2 Socket AMD EPYC 7713 64-Core Processor` with possible NUMA configuration such as 1, 2 and 4. ``` $ lstopo-no-graphics --merge Machine (503GB total) Package L#0 Group0 L#0 NUMANode L#0 (P#0 126GB) L3 L#0 (32MB) Core L#0 PU L#0 (P#0) PU L#1 (P#128) Core L#1 PU L#2 (P#1) PU L#3 (P#129) Core L#2 PU L#4 (P#2) PU L#5 (P#130) Core L#3 PU L#6 (P#3) PU L#7 (P#131) Core L#4 PU L#8 (P#4) PU L#9 (P#132) Core L#5 PU L#10 (P#5) PU L#11 (P#133) Core L#6 PU L#12 (P#6) PU L#13 (P#134) Core L#7 PU L#14 (P#7) PU L#15 (P#135) L3 L#1 (32MB) Core L#8 PU L#16 (P#8) PU L#17 (P#136) Core L#9 PU L#18 (P#9) PU L#19 (P#137) Core L#10 PU L#20 (P#10) PU L#21 (P#138) Core L#11 PU L#22 (P#11) PU L#23 (P#139) Core L#12 PU L#24 (P#12) PU L#25 (P#140) Core L#13 PU L#26 (P#13) PU L#27 (P#141) Core L#14 PU L#28 (P#14) PU L#29 (P#142) Core L#15 PU L#30 (P#15) PU L#31 (P#143) L3 L#2 (32MB) Core L#16 PU L#32 (P#16) PU L#33 (P#144) Core L#17 PU L#34 (P#17) PU L#35 (P#145) Core L#18 PU L#36 (P#18) PU L#37 (P#146) Core L#19 PU L#38 (P#19) PU L#39 (P#147) Core L#20 PU L#40 (P#20) PU L#41 (P#148) Core L#21 PU L#42 (P#21) PU L#43 (P#149) Core L#22 PU L#44 (P#22) PU L#45 (P#150) Core L#23 PU L#46 (P#23) PU L#47 (P#151) L3 L#3 (32MB) Core L#24 PU L#48 (P#24) PU L#49 (P#152) Core L#25 PU L#50 (P#25) PU L#51 (P#153) Core L#26 PU L#52 (P#26) PU L#53 (P#154) Core L#27 PU L#54 (P#27) PU L#55 (P#155) Core L#28 PU L#56 (P#28) PU L#57 (P#156) Core L#29 PU L#58 (P#29) PU L#59 (P#157) Core L#30 PU L#60 (P#30) PU L#61 (P#158) Core L#31 PU L#62 (P#31) PU L#63 (P#159) HostBridge PCIBridge PCI 41:00.0 (Ethernet) Net "ens1f0" OpenFabrics "mlx5_0" PCI 41:00.1 (Ethernet) Net "ens1f1" OpenFabrics "mlx5_1" HostBridge PCIBridge PCI 63:00.0 (Ethernet) Net "eno12399np0" PCI 63:00.1 (Ethernet) Net "eno12409np1" PCIBridge PCIBridge PCI 62:00.0 (VGA) Group0 L#1 NUMANode L#1 (P#1 126GB) L3 L#4 (32MB) Core L#32 PU L#64 (P#32) PU L#65 (P#160) Core L#33 PU L#66 (P#33) PU L#67 (P#161) Core L#34 PU L#68 (P#34) PU L#69 (P#162) Core L#35 PU L#70 (P#35) PU L#71 (P#163) Core L#36 PU L#72 (P#36) PU L#73 (P#164) Core L#37 PU L#74 (P#37) PU L#75 (P#165) Core L#38 PU L#76 (P#38) PU L#77 (P#166) Core L#39 PU L#78 (P#39) PU L#79 (P#167) L3 L#5 (32MB) Core L#40 PU L#80 (P#40) PU L#81 (P#168) Core L#41 PU L#82 (P#41) PU L#83 (P#169) Core L#42 PU L#84 (P#42) PU L#85 (P#170) Core L#43 PU L#86 (P#43) PU L#87 (P#171) Core L#44 PU L#88 (P#44) PU L#89 (P#172) Core L#45 PU L#90 (P#45) PU L#91 (P#173) Core L#46 PU L#92 (P#46) PU L#93 (P#174) Core L#47 PU L#94 (P#47) PU L#95 (P#175) L3 L#6 (32MB) Core L#48 PU L#96 (P#48) PU L#97 (P#176) Core L#49 PU L#98 (P#49) PU L#99 (P#177) Core L#50 PU L#100 (P#50) PU L#101 (P#178) Core L#51 PU L#102 (P#51) PU L#103 (P#179) Core L#52 PU L#104 (P#52) PU L#105 (P#180) Core L#53 PU L#106 (P#53) PU L#107 (P#181) Core L#54 PU L#108 (P#54) PU L#109 (P#182) Core L#55 PU L#110 (P#55) PU L#111 (P#183) L3 L#7 (32MB) Core L#56 PU L#112 (P#56) PU L#113 (P#184) Core L#57 PU L#114 (P#57) PU L#115 (P#185) Core L#58 PU L#116 (P#58) PU L#117 (P#186) Core L#59 PU L#118 (P#59) PU L#119 (P#187) Core L#60 PU L#120 (P#60) PU L#121 (P#188) Core L#61 PU L#122 (P#61) PU L#123 (P#189) Core L#62 PU L#124 (P#62) PU L#125 (P#190) Core L#63 PU L#126 (P#63) PU L#127 (P#191) HostBridge PCIBridge PCI 01:00.0 (RAID) PCIBridge PCI 02:00.0 (SATA) Block(Disk) "sda" HostBridge PCIBridge PCI 22:00.0 (NVMExp) Block(Disk) "nvme0n1" Package L#1 Group0 L#2 NUMANode L#2 (P#2 126GB) L3 L#8 (32MB) Core L#64 PU L#128 (P#64) PU L#129 (P#192) Core L#65 PU L#130 (P#65) PU L#131 (P#193) Core L#66 PU L#132 (P#66) PU L#133 (P#194) Core L#67 PU L#134 (P#67) PU L#135 (P#195) Core L#68 PU L#136 (P#68) PU L#137 (P#196) Core L#69 PU L#138 (P#69) PU L#139 (P#197) Core L#70 PU L#140 (P#70) PU L#141 (P#198) Core L#71 PU L#142 (P#71) PU L#143 (P#199) L3 L#9 (32MB) Core L#72 PU L#144 (P#72) PU L#145 (P#200) Core L#73 PU L#146 (P#73) PU L#147 (P#201) Core L#74 PU L#148 (P#74) PU L#149 (P#202) Core L#75 PU L#150 (P#75) PU L#151 (P#203) Core L#76 PU L#152 (P#76) PU L#153 (P#204) Core L#77 PU L#154 (P#77) PU L#155 (P#205) Core L#78 PU L#156 (P#78) PU L#157 (P#206) Core L#79 PU L#158 (P#79) PU L#159 (P#207) L3 L#10 (32MB) Core L#80 PU L#160 (P#80) PU L#161 (P#208) Core L#81 PU L#162 (P#81) PU L#163 (P#209) Core L#82 PU L#164 (P#82) PU L#165 (P#210) Core L#83 PU L#166 (P#83) PU L#167 (P#211) Core L#84 PU L#168 (P#84) PU L#169 (P#212) Core L#85 PU L#170 (P#85) PU L#171 (P#213) Core L#86 PU L#172 (P#86) PU L#173 (P#214) Core L#87 PU L#174 (P#87) PU L#175 (P#215) L3 L#11 (32MB) Core L#88 PU L#176 (P#88) PU L#177 (P#216) Core L#89 PU L#178 (P#89) PU L#179 (P#217) Core L#90 PU L#180 (P#90) PU L#181 (P#218) Core L#91 PU L#182 (P#91) PU L#183 (P#219) Core L#92 PU L#184 (P#92) PU L#185 (P#220) Core L#93 PU L#186 (P#93) PU L#187 (P#221) Core L#94 PU L#188 (P#94) PU L#189 (P#222) Core L#95 PU L#190 (P#95) PU L#191 (P#223) HostBridge PCIBridge PCI c5:00.0 (SATA) HostBridge PCIBridge PCI e1:00.0 (Ethernet) Net "eno8303" PCI e1:00.1 (Ethernet) Net "eno8403" Group0 L#3 NUMANode L#3 (P#3 126GB) L3 L#12 (32MB) Core L#96 PU L#192 (P#96) PU L#193 (P#224) Core L#97 PU L#194 (P#97) PU L#195 (P#225) Core L#98 PU L#196 (P#98) PU L#197 (P#226) Core L#99 PU L#198 (P#99) PU L#199 (P#227) Core L#100 PU L#200 (P#100) PU L#201 (P#228) Core L#101 PU L#202 (P#101) PU L#203 (P#229) Core L#102 PU L#204 (P#102) PU L#205 (P#230) Core L#103 PU L#206 (P#103) PU L#207 (P#231) L3 L#13 (32MB) Core L#104 PU L#208 (P#104) PU L#209 (P#232) Core L#105 PU L#210 (P#105) PU L#211 (P#233) Core L#106 PU L#212 (P#106) PU L#213 (P#234) Core L#107 PU L#214 (P#107) PU L#215 (P#235) Core L#108 PU L#216 (P#108) PU L#217 (P#236) Core L#109 PU L#218 (P#109) PU L#219 (P#237) Core L#110 PU L#220 (P#110) PU L#221 (P#238) Core L#111 PU L#222 (P#111) PU L#223 (P#239) L3 L#14 (32MB) Core L#112 PU L#224 (P#112) PU L#225 (P#240) Core L#113 PU L#226 (P#113) PU L#227 (P#241) Core L#114 PU L#228 (P#114) PU L#229 (P#242) Core L#115 PU L#230 (P#115) PU L#231 (P#243) Core L#116 PU L#232 (P#116) PU L#233 (P#244) Core L#117 PU L#234 (P#117) PU L#235 (P#245) Core L#118 PU L#236 (P#118) PU L#237 (P#246) Core L#119 PU L#238 (P#119) PU L#239 (P#247) L3 L#15 (32MB) Core L#120 PU L#240 (P#120) PU L#241 (P#248) Core L#121 PU L#242 (P#121) PU L#243 (P#249) Core L#122 PU L#244 (P#122) PU L#245 (P#250) Core L#123 PU L#246 (P#123) PU L#247 (P#251) Core L#124 PU L#248 (P#124) PU L#249 (P#252) Core L#125 PU L#250 (P#125) PU L#251 (P#253) Core L#126 PU L#252 (P#126) PU L#253 (P#254) Core L#127 PU L#254 (P#127) PU L#255 (P#255) HostBridge PCIBridge PCI 81:00.0 (Ethernet) Net "ens3f0" OpenFabrics "mlx5_2" PCI 81:00.1 (Ethernet) Net "ens3f1" OpenFabrics "mlx5_3" ``` The current result gives ``` $ ./usertools/cpu_layout.py ====================================================================== Core and Socket Information (as reported by '/sys/devices/system/cpu') ====================================================================== cores = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63] numa nodes per socket = [0, 1] sockets = [0, 1] Socket 0 Numa 0 ------------------- Core 0 [0, 128] Core 1 [1, 129] Core 2 [2, 130] Core 3 [3, 131] Core 4 [4, 132] Core 5 [5, 133] Core 6 [6, 134] Core 7 [7, 135] Core 8 [8, 136] Core 9 [9, 137] Core 10 [10, 138] Core 11 [11, 139] Core 12 [12, 140] Core 13 [13, 141] Core 14 [14, 142] Core 15 [15, 143] Core 16 [16, 144] Core 17 [17, 145] Core 18 [18, 146] Core 19 [19, 147] Core 20 [20, 148] Core 21 [21, 149] Core 22 [22, 150] Core 23 [23, 151] Core 24 [24, 152] Core 25 [25, 153] Core 26 [26, 154] Core 27 [27, 155] Core 28 [28, 156] Core 29 [29, 157] Core 30 [30, 158] Core 31 [31, 159] Socket 0 Numa 1 ------------------- Core 0 [32, 160] Core 1 [33, 161] Core 2 [34, 162] Core 3 [35, 163] Core 4 [36, 164] Core 5 [37, 165] Core 6 [38, 166] Core 7 [39, 167] Core 8 [40, 168] Core 9 [41, 169] Core 10 [42, 170] Core 11 [43, 171] Core 12 [44, 172] Core 13 [45, 173] Core 14 [46, 174] Core 15 [47, 175] Core 16 [48, 176] Core 17 [49, 177] Core 18 [50, 178] Core 19 [51, 179] Core 20 [52, 180] Core 21 [53, 181] Core 22 [54, 182] Core 23 [55, 183] Core 24 [56, 184] Core 25 [57, 185] Core 26 [58, 186] Core 27 [59, 187] Core 28 [60, 188] Core 29 [61, 189] Core 30 [62, 190] Core 31 [63, 191] Socket 1 Numa 2 ------------------- Core 0 [64, 192] Core 1 [65, 193] Core 2 [66, 194] Core 3 [67, 195] Core 4 [68, 196] Core 5 [69, 197] Core 6 [70, 198] Core 7 [71, 199] Core 8 [72, 200] Core 9 [73, 201] Core 10 [74, 202] Core 11 [75, 203] Core 12 [76, 204] Core 13 [77, 205] Core 14 [78, 206] Core 15 [79, 207] Core 16 [80, 208] Core 17 [81, 209] Core 18 [82, 210] Core 19 [83, 211] Core 20 [84, 212] Core 21 [85, 213] Core 22 [86, 214] Core 23 [87, 215] Core 24 [88, 216] Core 25 [89, 217] Core 26 [90, 218] Core 27 [91, 219] Core 28 [92, 220] Core 29 [93, 221] Core 30 [94, 222] Core 31 [95, 223] Socket 1 Numa 3 ------------------- Core 0 [96, 224] Core 1 [97, 225] Core 2 [98, 226] Core 3 [99, 227] Core 4 [100, 228] Core 5 [101, 229] Core 6 [102, 230] Core 7 [103, 231] Core 8 [104, 232] Core 9 [105, 233] Core 10 [106, 234] Core 11 [107, 235] Core 12 [108, 236] Core 13 [109, 237] Core 14 [110, 238] Core 15 [111, 239] Core 16 [112, 240] Core 17 [113, 241] Core 18 [114, 242] Core 19 [115, 243] Core 20 [116, 244] Core 21 [117, 245] Core 22 [118, 246] Core 23 [119, 247] Core 24 [120, 248] Core 25 [121, 249] Core 26 [122, 250] Core 27 [123, 251] Core 28 [124, 252] Core 29 [125, 253] Core 30 [126, 254] Core 31 [127, 255] ```