On Wed, 27 Jan 2021 15:31:40 -0500 Michael Di Domenico via users <users@lists.open-mpi.org> wrote:
> if you have OPA cards, for openmpi you only need --with-ofi, you don't > need psm/psm2/verbs/ucx. I agree with Michael and would add for clarity that on the system you always need PSM2 and optionally libfabric (if you go the ofi way with OpenMPI). In OpenMPI you need EITHER the PSM2 mtl or the ofi mtl. Ending up with a mix can cause issues. Note that the existence of libfabric on the system will build the ofi mtl (as if you gave --with-ofi). On our OPA clusters we run only the psm2 mtl but this is on OpenMPI-3.x. Verbs (openib btl) and ucx should be avoided on OPA as should PSM which has nothing to do with OPA (it is for Truescale). /Peter K
