So, just to be sure - when you run 320 "cores", you are running across 20 nodes?
Just want to ensure we are using "core" the same way - some people confuse cores with hyperthreads. On Jun 9, 2013, at 3:50 PM, "Blosch, Edwin L" <edwin.l.blo...@lmco.com> wrote: > 16. dual-socket Xeon, E5-2670. > > I am trying a larger model to see if the performance drop-off happens at a > different number of cores. > Also I’m running some intermediate core-count sizes to refine the curve a bit. > I also added mpi_show_mca_params all, and at the same time, > btl_openib_use_eager_rdma 1, just to see if that does anything. > > From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On > Behalf Of Ralph Castain > Sent: Sunday, June 09, 2013 5:04 PM > To: Open MPI Users > Subject: EXTERNAL: Re: [OMPI users] Need advice on performance problem > > Looks to me like things are okay thru 160, and then things fall apart after > that point. How many cores are on a node? > > > On Jun 9, 2013, at 1:59 PM, "Blosch, Edwin L" <edwin.l.blo...@lmco.com> wrote: > > > I’m having some trouble getting good scaling with OpenMPI 1.6.4 and I don’t > know where to start looking. This is an Infiniband FDR network with Sandy > Bridge nodes. I am using affinity (--bind-to-core) but no other options. As > the number of cores goes up, the message sizes are typically going down. > There seem to be lots of options in the FAQ, and I would welcome any advice > on where to start. All these timings are on a completely empty system except > for me. > > Thanks > > > MPI # cores Ave. Rate Std. Dev. % # timings Speedup > Efficiency > ================================================================================================ > MVAPICH | 16 | 8.6783 | 0.995 % | 2 | 16.000 | > 1.0000 > MVAPICH | 48 | 8.7665 | 1.937 % | 3 | 47.517 | > 0.9899 > MVAPICH | 80 | 8.8900 | 2.291 % | 3 | 78.095 | > 0.9762 > MVAPICH | 160 | 8.9897 | 2.409 % | 3 | 154.457 | > 0.9654 > MVAPICH | 320 | 8.9780 | 2.801 % | 3 | 309.317 | > 0.9666 > MVAPICH | 480 | 8.9704 | 2.316 % | 3 | 464.366 | > 0.9674 > MVAPICH | 640 | 9.0792 | 1.138 % | 3 | 611.739 | > 0.9558 > MVAPICH | 720 | 9.1328 | 1.052 % | 3 | 684.162 | > 0.9502 > MVAPICH | 800 | 9.1945 | 0.773 % | 3 | 755.079 | > 0.9438 > OpenMPI | 16 | 8.6743 | 2.335 % | 2 | 16.000 | > 1.0000 > OpenMPI | 48 | 8.7826 | 1.605 % | 2 | 47.408 | > 0.9877 > OpenMPI | 80 | 8.8861 | 0.120 % | 2 | 78.093 | > 0.9762 > OpenMPI | 160 | 8.9774 | 0.785 % | 2 | 154.598 | > 0.9662 > OpenMPI | 320 | 12.0585 | 16.950 % | 2 | 230.191 | > 0.7193 > OpenMPI | 480 | 14.8330 | 1.300 % | 2 | 280.701 | > 0.5848 > OpenMPI | 640 | 17.1723 | 2.577 % | 3 | 323.283 | > 0.5051 > OpenMPI | 720 | 18.2153 | 2.798 % | 3 | 342.868 | > 0.4762 > OpenMPI | 800 | 19.3603 | 2.254 % | 3 | 358.434 | > 0.4480 > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
