I'll try to take a look at it - my expectation is that lama might get stuck because you didn't tell it a pattern to map, and I doubt that code path has seen much testing.
On Dec 20, 2013, at 5:52 PM, tmish...@jcity.maeda.co.jp wrote: > > > Hi Ralph, I'm glad to hear that, thanks. > > By the way, yesterday I tried to check how lama in 1.7.4rc treat numa node. > > Then, even wiht this simple command line, it freezed without any massage: > > mpirun -np 2 -host node05 -mca rmaps lama myprog > > Could you check what happened? > > Is it better to open new thread or continue this thread? > > Regards, > Tetsuya Mishima > > >> I'll make it work so that NUMA can be either above or below socket >> >> On Dec 20, 2013, at 2:57 AM, tmish...@jcity.maeda.co.jp wrote: >> >>> >>> >>> Hi Brice, >>> >>> Thank you for your comment. I understand what you mean. >>> >>> My opinion was made just considering easy way to adjust the code for >>> inversion of hierarchy in object tree. >>> >>> Tetsuya Mishima >>> >>> >>>> I don't think there's any such difference. >>>> Also, all these NUMA architectures are reported the same by hwloc, and >>>> therefore used the same in Open MPI. >>>> >>>> And yes, L3 and NUMA are topologically-identical on AMD Magny-Cours > (and >>>> most recent AMD and Intel platforms). >>>> >>>> Brice >>>> >>>> >>>> >>>> Le 20/12/2013 11:33, tmish...@jcity.maeda.co.jp a écrit : >>>>> >>>>> Hi Ralph, >>>>> >>>>> The numa-node in AMD Mangy-Cours/Interlagos is so called cc(cache >>>>> coherent)NUMA, >>>>> which seems to be a little bit different from the traditional numa >>> defined >>>>> in openmpi. >>>>> >>>>> I notice that ccNUMA object is almost same as L3cache object. >>>>> So "-bind-to l3cache" or "-map-by l3cache" is valid for what I want > to >>> do. >>>>> Therefore, "do not touch it" is one of the solution, I think ... >>>>> >>>>> Anyway, mixing up these two types of numa is the problem. >>>>> >>>>> Regards, >>>>> Tetsuya Mishima >>>>> >>>>>> I can wait it'll be fixed in 1.7.5 or later, because putting > "-bind-to >>>>>> numa" >>>>>> and "-map-by numa" at the same time works as a workaround. >>>>>> >>>>>> Thanks, >>>>>> Tetsuya Mishima >>>>>> >>>>>>> Yeah, it will impact everything that uses hwloc topology maps, I >>> fear. >>>>>>> >>>>>>> One side note: you'll need to add --hetero-nodes to your cmd line. > If >>>>> we >>>>>> don't see that, we assume that all the node topologies are identical > - >>>>>> which clearly isn't true here. >>>>>>> I'll try to resolve the hier inversion over the holiday - won't be >>> for >>>>>> 1.7.4, but hopefully for 1.7.5 >>>>>>> Thanks >>>>>>> Ralph >>>>>>> >>>>>>> On Dec 18, 2013, at 9:44 PM, tmish...@jcity.maeda.co.jp wrote: >>>>>>> >>>>>>>> >>>>>>>> I think it's normal for AMD opteron having 8/16 cores such as >>>>>>>> magny cours or interlagos. Because it usually has 2 numa nodes >>>>>>>> in a cpu(socket), numa-node can not include a socket. This type >>>>>>>> of hierarchy would be natural. >>>>>>>> >>>>>>>> (node03 is Dell PowerEdge R815 and maybe quite common, I guess) >>>>>>>> >>>>>>>> By the way, I think this inversion should affect rmaps_lama > mapping. >>>>>>>> >>>>>>>> Tetsuya Mishima >>>>>>>> >>>>>>>>> Ick - yeah, that would be a problem. I haven't seen that type of >>>>>>>> hierarchical inversion before - is node03 a different type of > chip? >>>>>>>>> Might take awhile for me to adjust the code to handle hier >>>>>>>> inversion... :-( >>>>>>>>> On Dec 18, 2013, at 9:05 PM, tmish...@jcity.maeda.co.jp wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Ralph, >>>>>>>>>> >>>>>>>>>> I found the reason. I attached the main part of output with 32 >>>>>>>>>> core node(node03) and 8 core node(node05) at the bottom. >>>>>>>>>> >>>>>>>>>> From this information, socket of node03 includes numa-node. >>>>>>>>>> On the other hand, numa-node of node05 includes socket. >>>>>>>>>> The direction of object tree is opposite. >>>>>>>>>> >>>>>>>>>> Since "-map-by socket" may be assumed as default, >>>>>>>>>> for node05, "-bind-to numa and -map-by socket" means >>>>>>>>>> upward search. For node03, this should be downward. >>>>>>>>>> >>>>>>>>>> I guess that openmpi-1.7.4rc1 will always assume numa-node >>>>>>>>>> includes socket. Is it right? Then, upward search is assumed >>>>>>>>>> in orte_rmaps_base_compute_bindings even for node03 when I >>>>>>>>>> put "-bind-to numa and -map-by socket" option. >>>>>>>>>> >>>>>>>>>> [node03.cluster:15508] [[38286,0],0] rmaps:base:compute_usage >>>>>>>>>> [node03.cluster:15508] mca:rmaps: compute bindings for job >>>>> [38286,1] >>>>>>>> with >>>>>>>>>> policy NUMA >>>>>>>>>> [node03.cluster:15508] mca:rmaps: bind upwards for job [38286,1] >>>>> with >>>>>>>>>> bindings NUMA >>>>>>>>>> [node03.cluster:15508] [[38286,0],0] bind:upward target NUMANode >>>>> type >>>>>>>>>> Machine >>>>>>>>>> >>>>>>>>>> That's the reason of this trouble. Therefore, adding "-map-by >>> core" >>>>>>>> works. >>>>>>>>>> (mapping pattern seems to be strange ...) >>>>>>>>>> >>>>>>>>>> [mishima@node03 demos]$ mpirun -np 8 -bind-to numa -map-by core >>>>>>>>>> -report-bindings myprog >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>>>> NUMANode >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>>>> NUMANode >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>>>> NUMANode >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>>>> NUMANode >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>>>> NUMANode >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>>>> NUMANode >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>>>> NUMANode >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>> Cache >>>>>>>>>> [node03.cluster:15885] [[38679,0],0] bind:upward target NUMANode >>>>> type >>>>>>>>>> NUMANode >>>>>>>>>> [node03.cluster:15885] MCW rank 2 bound to socket 0[core 0[hwt >>> 0]], >>>>>>>> socket >>>>>>>>>> 0[core 1[hwt 0]], socket 0[core 2[hwt 0]], so >>>>>>>>>> cket 0[core 3[hwt 0]]: >>>>>>>>>> >>>>> [B/B/B/B/./././.][./././././././.][./././././././.][./././././././.] >>>>>>>>>> [node03.cluster:15885] MCW rank 3 bound to socket 0[core 0[hwt >>> 0]], >>>>>>>> socket >>>>>>>>>> 0[core 1[hwt 0]], socket 0[core 2[hwt 0]], so >>>>>>>>>> cket 0[core 3[hwt 0]]: >>>>>>>>>> >>>>> [B/B/B/B/./././.][./././././././.][./././././././.][./././././././.] >>>>>>>>>> [node03.cluster:15885] MCW rank 4 bound to socket 0[core 4[hwt >>> 0]], >>>>>>>> socket >>>>>>>>>> 0[core 5[hwt 0]], socket 0[core 6[hwt 0]], so >>>>>>>>>> cket 0[core 7[hwt 0]]: >>>>>>>>>> >>>>> [././././B/B/B/B][./././././././.][./././././././.][./././././././.] >>>>>>>>>> [node03.cluster:15885] MCW rank 5 bound to socket 0[core 4[hwt >>> 0]], >>>>>>>> socket >>>>>>>>>> 0[core 5[hwt 0]], socket 0[core 6[hwt 0]], so >>>>>>>>>> cket 0[core 7[hwt 0]]: >>>>>>>>>> >>>>> [././././B/B/B/B][./././././././.][./././././././.][./././././././.] >>>>>>>>>> [node03.cluster:15885] MCW rank 6 bound to socket 0[core 4[hwt >>> 0]], >>>>>>>> socket >>>>>>>>>> 0[core 5[hwt 0]], socket 0[core 6[hwt 0]], so >>>>>>>>>> cket 0[core 7[hwt 0]]: >>>>>>>>>> >>>>> [././././B/B/B/B][./././././././.][./././././././.][./././././././.] >>>>>>>>>> [node03.cluster:15885] MCW rank 7 bound to socket 0[core 4[hwt >>> 0]], >>>>>>>> socket >>>>>>>>>> 0[core 5[hwt 0]], socket 0[core 6[hwt 0]], so >>>>>>>>>> cket 0[core 7[hwt 0]]: >>>>>>>>>> >>>>> [././././B/B/B/B][./././././././.][./././././././.][./././././././.] >>>>>>>>>> [node03.cluster:15885] MCW rank 0 bound to socket 0[core 0[hwt >>> 0]], >>>>>>>> socket >>>>>>>>>> 0[core 1[hwt 0]], socket 0[core 2[hwt 0]], so >>>>>>>>>> cket 0[core 3[hwt 0]]: >>>>>>>>>> >>>>> [B/B/B/B/./././.][./././././././.][./././././././.][./././././././.] >>>>>>>>>> [node03.cluster:15885] MCW rank 1 bound to socket 0[core 0[hwt >>> 0]], >>>>>>>> socket >>>>>>>>>> 0[core 1[hwt 0]], socket 0[core 2[hwt 0]], so >>>>>>>>>> cket 0[core 3[hwt 0]]: >>>>>>>>>> >>>>> [B/B/B/B/./././.][./././././././.][./././././././.][./././././././.] >>>>>>>>>> Hello world from process 6 of 8 >>>>>>>>>> Hello world from process 5 of 8 >>>>>>>>>> Hello world from process 0 of 8 >>>>>>>>>> Hello world from process 7 of 8 >>>>>>>>>> Hello world from process 3 of 8 >>>>>>>>>> Hello world from process 4 of 8 >>>>>>>>>> Hello world from process 2 of 8 >>>>>>>>>> Hello world from process 1 of 8 >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Tetsuya Mishima >>>>>>>>>> >>>>>>>>>> [node03.cluster:15508] Type: Machine Number of child objects: 4 >>>>>>>>>> Name=NULL >>>>>>>>>> total=132358820KB >>>>>>>>>> Backend=Linux >>>>>>>>>> OSName=Linux >>>>>>>>>> OSRelease=2.6.18-308.16.1.el5 >>>>>>>>>> OSVersion="#1 SMP Tue Oct 2 22:01:43 EDT 2012" >>>>>>>>>> Architecture=x86_64 >>>>>>>>>> Cpuset: 0xffffffff >>>>>>>>>> Online: 0xffffffff >>>>>>>>>> Allowed: 0xffffffff >>>>>>>>>> Bind CPU proc: TRUE >>>>>>>>>> Bind CPU thread: TRUE >>>>>>>>>> Bind MEM proc: FALSE >>>>>>>>>> Bind MEM thread: TRUE >>>>>>>>>> Type: Socket Number of child objects: 2 >>>>>>>>>> Name=NULL >>>>>>>>>> total=33071780KB >>>>>>>>>> CPUModel="AMD Opteron(tm) Processor 6136" >>>>>>>>>> Cpuset: 0x000000ff >>>>>>>>>> Online: 0x000000ff >>>>>>>>>> Allowed: 0x000000ff >>>>>>>>>> Type: NUMANode Number of child objects: 1 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [node05.cluster:21750] Type: Machine Number of child objects: 2 >>>>>>>>>> Name=NULL >>>>>>>>>> total=33080072KB >>>>>>>>>> Backend=Linux >>>>>>>>>> OSName=Linux >>>>>>>>>> OSRelease=2.6.18-308.16.1.el5 >>>>>>>>>> OSVersion="#1 SMP Tue Oct 2 22:01:43 EDT 2012" >>>>>>>>>> Architecture=x86_64 >>>>>>>>>> Cpuset: 0x000000ff >>>>>>>>>> Online: 0x000000ff >>>>>>>>>> Allowed: 0x000000ff >>>>>>>>>> Bind CPU proc: TRUE >>>>>>>>>> Bind CPU thread: TRUE >>>>>>>>>> Bind MEM proc: FALSE >>>>>>>>>> Bind MEM thread: TRUE >>>>>>>>>> Type: NUMANode Number of child objects: 1 >>>>>>>>>> Name=NULL >>>>>>>>>> local=16532232KB >>>>>>>>>> total=16532232KB >>>>>>>>>> Cpuset: 0x0000000f >>>>>>>>>> Online: 0x0000000f >>>>>>>>>> Allowed: 0x0000000f >>>>>>>>>> Type: Socket Number of child objects: 1 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Hmm...try adding "-mca rmaps_base_verbose 10 -mca >>> ess_base_verbose >>>>>> 5" >>>>>>>> to >>>>>>>>>> your cmd line and let's see what it thinks it found. >>>>>>>>>>> >>>>>>>>>>> On Dec 18, 2013, at 6:55 PM, tmish...@jcity.maeda.co.jp wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi, I report one more problem with openmpi-1.7.4rc1, >>>>>>>>>>>> which is more serious. >>>>>>>>>>>> >>>>>>>>>>>> For our 32 core nodes(AMD magny cours based) which has >>>>>>>>>>>> 8 numa-nodes, "-bind-to numa" does not work. Without >>>>>>>>>>>> this option, it works. For your infomation, at the >>>>>>>>>>>> bottom of this mail, I added the lstopo information >>>>>>>>>>>> of the node. >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Tetsuya Mishima >>>>>>>>>>>> >>>>>>>>>>>> [mishima@manage ~]$ qsub -I -l nodes=1:ppn=32 >>>>>>>>>>>> qsub: waiting for job 8352.manage.cluster to start >>>>>>>>>>>> qsub: job 8352.manage.cluster ready >>>>>>>>>>>> >>>>>>>>>>>> [mishima@node03 demos]$ mpirun -np 8 -report-bindings -bind-to >>>>> numa >>>>>>>>>> myprog >>>>>>>>>>>> [node03.cluster:15316] [[37582,0],0] bind:upward target > NUMANode >>>>>> type >>>>>>>>>>>> Machine >>>>>>>>>>>> >>>>> >>> > -------------------------------------------------------------------------- >>>>>>>>>>>> A request was made to bind to NUMA, but an appropriate target >>>>> could >>>>>>>> not >>>>>>>>>>>> be found on node node03. >>>>>>>>>>>> >>>>> >>> > -------------------------------------------------------------------------- >>>>>>>>>>>> [mishima@node03 ~]$ cd ~/Desktop/openmpi-1.7/demos/ >>>>>>>>>>>> [mishima@node03 demos]$ mpirun -np 8 -report-bindings myprog >>>>>>>>>>>> [node03.cluster:15282] MCW rank 2 bound to socket 1[core 8[hwt >>>>> 0]]: >>>>>>>>>>>> [./././././././.][B/././././././.][./././././././.][ >>>>>>>>>>>> ./././././././.] >>>>>>>>>>>> [node03.cluster:15282] MCW rank 3 bound to socket 1[core 9[hwt >>>>> 0]]: >>>>>>>>>>>> [./././././././.][./B/./././././.][./././././././.][ >>>>>>>>>>>> ./././././././.] >>>>>>>>>>>> [node03.cluster:15282] MCW rank 4 bound to socket 2[core 16 > [hwt >>>>>> 0]]: >>>>>>>>>>>> [./././././././.][./././././././.][B/././././././.] >>>>>>>>>>>> [./././././././.] >>>>>>>>>>>> [node03.cluster:15282] MCW rank 5 bound to socket 2[core 17 > [hwt >>>>>> 0]]: >>>>>>>>>>>> [./././././././.][./././././././.][./B/./././././.] >>>>>>>>>>>> [./././././././.] >>>>>>>>>>>> [node03.cluster:15282] MCW rank 6 bound to socket 3[core 24 > [hwt >>>>>> 0]]: >>>>>>>>>>>> [./././././././.][./././././././.][./././././././.] >>>>>>>>>>>> [B/././././././.] >>>>>>>>>>>> [node03.cluster:15282] MCW rank 7 bound to socket 3[core 25 > [hwt >>>>>> 0]]: >>>>>>>>>>>> [./././././././.][./././././././.][./././././././.] >>>>>>>>>>>> [./B/./././././.] >>>>>>>>>>>> [node03.cluster:15282] MCW rank 0 bound to socket 0[core 0[hwt >>>>> 0]]: >>>>>>>>>>>> [B/././././././.][./././././././.][./././././././.][ >>>>>>>>>>>> ./././././././.] >>>>>>>>>>>> [node03.cluster:15282] MCW rank 1 bound to socket 0[core 1[hwt >>>>> 0]]: >>>>>>>>>>>> [./B/./././././.][./././././././.][./././././././.][ >>>>>>>>>>>> ./././././././.] >>>>>>>>>>>> Hello world from process 2 of 8 >>>>>>>>>>>> Hello world from process 5 of 8 >>>>>>>>>>>> Hello world from process 4 of 8 >>>>>>>>>>>> Hello world from process 3 of 8>>>>>>>>>> Hello world from > process 1 of 8 >>>>>>>>>>>> Hello world from process 7 of 8 >>>>>>>>>>>> Hello world from process 6 of 8 >>>>>>>>>>>> Hello world from process 0 of 8 >>>>>>>>>>>> [mishima@node03 demos]$ ~/opt/hwloc/bin/lstopo-no-graphics >>>>>>>>>>>> Machine (126GB) >>>>>>>>>>>> Socket L#0 (32GB) >>>>>>>>>>>> NUMANode L#0 (P#0 16GB) + L3 L#0 (5118KB) >>>>>>>>>>>> L2 L#0 (512KB) + L1d L#0 (64KB) + L1i L#0 (64KB) + Core L#0 > + >>>>> PU >>>>>>>>>> L#0 >>>>>>>>>>>> (P#0) >>>>>>>>>>>> L2 L#1 (512KB) + L1d L#1 (64KB) + L1i L#1 (64KB) + Core L#1 > + >>>>> PU >>>>>>>>>> L#1 >>>>>>>>>>>> (P#1) >>>>>>>>>>>> L2 L#2 (512KB) + L1d L#2 (64KB) + L1i L#2 (64KB) + Core L#2 > + >>>>> PU >>>>>>>>>> L#2 >>>>>>>>>>>> (P#2) >>>>>>>>>>>> L2 L#3 (512KB) + L1d L#3 (64KB) + L1i L#3 (64KB) + Core L#3 > + >>>>> PU >>>>>>>>>> L#3 >>>>>>>>>>>> (P#3) >>>>>>>>>>>> NUMANode L#1 (P#1 16GB) + L3 L#1 (5118KB) >>>>>>>>>>>> L2 L#4 (512KB) + L1d L#4 (64KB) + L1i L#4 (64KB) + Core L#4 > + >>>>> PU >>>>>>>>>> L#4 >>>>>>>>>>>> (P#4) >>>>>>>>>>>> L2 L#5 (512KB) + L1d L#5 (64KB) + L1i L#5 (64KB) + Core L#5 > + >>>>> PU >>>>>>>>>> L#5 >>>>>>>>>>>> (P#5) >>>>>>>>>>>> L2 L#6 (512KB) + L1d L#6 (64KB) + L1i L#6 (64KB) + Core L#6 > + >>>>> PU >>>>>>>>>> L#6 >>>>>>>>>>>> (P#6) >>>>>>>>>>>> L2 L#7 (512KB) + L1d L#7 (64KB) + L1i L#7 (64KB) + Core L#7 > + >>>>> PU>>>>>> L#7 >>>>>>>>>>>> (P#7) >>>>>>>>>>>> Socket L#1 (32GB) >>>>>>>>>>>> NUMANode L#2 (P#6 16GB) + L3 L#2 (5118KB) >>>>>>>>>>>> L2 L#8 (512KB) + L1d L#8 (64KB) + L1i L#8 (64KB) + Core L#8 > + >>>>> PU >>>>>>>>>> L#8 >>>>>>>>>>>> (P#8) >>>>>>>>>>>> L2 L#9 (512KB) + L1d L#9 (64KB) + L1i L#9 (64KB) + Core L#9 > + >>>>> PU >>>>>>>>>> L#9 >>>>>>>>>>>> (P#9) >>>>>>>>>>>> L2 L#10 (512KB) + L1d L#10 (64KB) + L1i L#10 (64KB) + Core >>>>> L#10 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#10 (P#10) >>>>>>>>>>>> L2 L#11 (512KB) + L1d L#11 (64KB) + L1i L#11 (64KB) + Core >>>>> L#11 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#11 (P#11) >>>>>>>>>>>> NUMANode L#3 (P#7 16GB) + L3 L#3 (5118KB) >>>>>>>>>>>> L2 L#12 (512KB) + L1d L#12 (64KB) + L1i L#12 (64KB) + Core >>>>> L#12 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#12 (P#12) >>>>>>>>>>>> L2 L#13 (512KB) + L1d L#13 (64KB) + L1i L#13 (64KB) + Core >>>>> L#13 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#13 (P#13) >>>>>>>>>>>> L2 L#14 (512KB) + L1d L#14 (64KB) + L1i L#14 (64KB) + Core >>>>> L#14 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#14 (P#14) >>>>>>>>>>>> L2 L#15 (512KB) + L1d L#15 (64KB) + L1i L#15 (64KB) + Core >>>>> L#15 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#15 (P#15) >>>>>>>>>>>> Socket L#2 (32GB) >>>>>>>>>>>> NUMANode L#4 (P#4 16GB) + L3 L#4 (5118KB) >>>>>>>>>>>> L2 L#16 (512KB) + L1d L#16 (64KB) + L1i L#16 (64KB) + Core >>>>> L#16 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#16 (P#16) >>>>>>>>>>>> L2 L#17 (512KB) + L1d L#17 (64KB) + L1i L#17 (64KB) + Core >>>>> L#17 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#17 (P#17)> >>>>> L2 L#18 (512KB) + L1d L#18 (64KB) + L1i >>>>> L#18 (64KB) + Core L#18 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#18 (P#18) >>>>>>>>>>>> L2 L#19 (512KB) + L1d L#19 (64KB) + L1i L#19 (64KB) + Core >>>>> L#19 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#19 (P#19) >>>>>>>>>>>> NUMANode L#5 (P#5 16GB) + L3 L#5 (5118KB) >>>>>>>>>>>> L2 L#20 (512KB) + L1d L#20 (64KB) + L1i L#20 (64KB) + Core >>>>> L#20 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#20 (P#20) >>>>>>>>>>>> L2 L#21 (512KB) + L1d L#21 (64KB) + L1i L#21 (64KB) + Core >>>>> L#21 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#21 (P#21) >>>>>>>>>>>> L2 L#22 (512KB) + L1d L#22 (64KB) + L1i L#22 (64KB) + Core >>>>> L#22 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#22 (P#22) >>>>>>>>>>>> L2 L#23 (512KB) + L1d L#23 (64KB) + L1i L#23 (64KB) + Core >>>>> L#23 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#23 (P#23) >>>>>>>>>>>> Socket L#3 (32GB) >>>>>>>>>>>> NUMANode L#6 (P#2 16GB) + L3 L#6 (5118KB) >>>>>>>>>>>> L2 L#24 (512KB) + L1d L#24 (64KB) + L1i L#24 (64KB) + Core >>>>> L#24 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#24 (P#24)>>>>> L2 L#25 (512KB) + L1d L#25 (64KB) + L1i > L#25 >>>>>> (64KB) + Core L#25 + >>>>>>>>>> PU >>>>>>>>>>>> L#25 (P#25) >>>>>>>>>>>> L2 L#26 (512KB) + L1d L#26 (64KB) + L1i L#26 (64KB) + Core >>>>> L#26 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#26 (P#26) >>>>>>>>>>>> L2 L#27 (512KB) + L1d L#27 (64KB) + L1i L#27 (64KB) + Core >>>>> L#27 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#27 (P#27) >>>>>>>>>>>> NUMANode L#7 (P#3 16GB) + L3 L#7 (5118KB) >>>>>>>>>>>> L2 L#28 (512KB) + L1d L#28 (64KB) + L1i L#28 (64KB) + Core >>>>> L#28 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#28 (P#28) >>>>>>>>>>>> L2 L#29 (512KB) + L1d L#29 (64KB) + L1i L#29 (64KB) + Core >>>>> L#29 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#29 (P#29) >>>>>>>>>>>> L2 L#30 (512KB) + L1d L#30 (64KB) + L1i L#30 (64KB) + Core >>>>> L#30 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#30 (P#30) >>>>>>>>>>>> L2 L#31 (512KB) + L1d L#31 (64KB) + L1i L#31 (64KB) + Core >>>>> L#31 >>>>>> + >>>>>>>>>> PU >>>>>>>>>>>> L#31 (P#31) >>>>>>>>>>>> HostBridge L#0 >>>>>>>>>>>> PCIBridge >>>>>>>>>>>> PCI 14e4:1639 >>>>>>>>>>>> Net L#0 "eth0" >>>>>>>>>>>> PCI 14e4:1639 >>>>>>>>>>>> Net L#1 "eth1" >>>>>>>>>>>> PCIBridge >>>>>>>>>>>> PCI 14e4:1639 >>>>>>>>>>>> Net L#2 "eth2" >>>>>>>>>>>> PCI 14e4:1639 >>>>>>>>>>>> Net L#3 "eth3" >>>>>>>>>>>> PCIBridge >>>>>>>>>>>> PCIBridge >>>>>>>>>>>> PCIBridge >>>>>>>>>>>> PCI 1000:0072 >>>>>>>>>>>> Block L#4 "sdb" >>>>>>>>>>>> Block L#5 "sda" >>>>>>>>>>>> PCI 1002:4390 >>>>>>>>>>>> Block L#6 "sr0" >>>>>>>>>>>> PCIBridge >>>>>>>>>>>> PCI 102b:0532 >>>>>>>>>>>> HostBridge L#7 >>>>>>>>>>>> PCIBridge >>>>>>>>>>>> PCI 15b3:6274 >>>>>>>>>>>> Net L#7 "ib0" >>>>>>>>>>>> OpenFabrics L#8 "mthca0" >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> users mailing list >>>>>>>>>>>> us...@open-mpi.org >>>>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> users mailing list >>>>>>>>>>> us...@open-mpi.org>> >>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>>>>>>> _______________________________________________ >>>>>>>>>> users mailing list >>>>>>>>>> us...@open-mpi.org >>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>>>>>> _______________________________________________ >>>>>>>>> users mailing list >>>>>>>>> us...@open-mpi.org >>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>>>>> _______________________________________________ >>>>>>>> users mailing list >>>>>>>> us...@open-mpi.org >>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>>>> _______________________________________________ >>>>>>> users mailing list >>>>>>> us...@open-mpi.org >>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>>> _______________________________________________ >>>>>> users mailing list >>>>>> us...@open-mpi.org >>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>> _______________________________________________ >>>>> users mailing list >>>>> us...@open-mpi.org >>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>>> >>>> _______________________________________________ >>>> users mailing list >>>> us...@open-mpi.org >>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>> >>> _______________________________________________ >>> users mailing list >>> us...@open-mpi.org >>> http://www.open-mpi.org/mailman/listinfo.cgi/users >> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/users > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
