I'm having some trouble getting good scaling with OpenMPI 1.6.4 and I don't
know where to start looking. This is an Infiniband FDR network with Sandy
Bridge nodes. I am using affinity (--bind-to-core) but no other options. As
the number of cores goes up, the message sizes are typically going down. There
seem to be lots of options in the FAQ, and I would welcome any advice on where
to start. All these timings are on a completely empty system except for me.
Thanks
MPI # cores Ave. Rate Std. Dev. % # timings Speedup
Efficiency
================================================================================================
MVAPICH | 16 | 8.6783 | 0.995 % | 2 | 16.000 |
1.0000
MVAPICH | 48 | 8.7665 | 1.937 % | 3 | 47.517 |
0.9899
MVAPICH | 80 | 8.8900 | 2.291 % | 3 | 78.095 |
0.9762
MVAPICH | 160 | 8.9897 | 2.409 % | 3 | 154.457 |
0.9654
MVAPICH | 320 | 8.9780 | 2.801 % | 3 | 309.317 |
0.9666
MVAPICH | 480 | 8.9704 | 2.316 % | 3 | 464.366 |
0.9674
MVAPICH | 640 | 9.0792 | 1.138 % | 3 | 611.739 |
0.9558
MVAPICH | 720 | 9.1328 | 1.052 % | 3 | 684.162 |
0.9502
MVAPICH | 800 | 9.1945 | 0.773 % | 3 | 755.079 |
0.9438
OpenMPI | 16 | 8.6743 | 2.335 % | 2 | 16.000 |
1.0000
OpenMPI | 48 | 8.7826 | 1.605 % | 2 | 47.408 |
0.9877
OpenMPI | 80 | 8.8861 | 0.120 % | 2 | 78.093 |
0.9762
OpenMPI | 160 | 8.9774 | 0.785 % | 2 | 154.598 |
0.9662
OpenMPI | 320 | 12.0585 | 16.950 % | 2 | 230.191 |
0.7193
OpenMPI | 480 | 14.8330 | 1.300 % | 2 | 280.701 |
0.5848
OpenMPI | 640 | 17.1723 | 2.577 % | 3 | 323.283 |
0.5051
OpenMPI | 720 | 18.2153 | 2.798 % | 3 | 342.868 |
0.4762
OpenMPI | 800 | 19.3603 | 2.254 % | 3 | 358.434 |
0.4480
