Re: [OMPI users] OpenMPI with PSM on True Scale with OmniPath drivers

2018-01-22 Thread Cabral, Matias A
Hi William, Couple other questions: - Please share how you ompi configure line looks like. - Please clarify which is/are the compat libraries you refer to. There are some that are actually for the opposite case: Making TS app/libs run on Omnipath. - As Gilles mentions, moving to a newer

Re: [OMPI users] OpenMPI with PSM on True Scale with OmniPath drivers

2018-01-22 Thread Gilles Gouaillardet
William, In order to force PSM (aka Infinipath) you can mpirun --mca pml cm --mca mtl psm ... (Replace with psm2 for PSM2 (aka Omnipath) You can also mpirun --mca pml_base_verbose 10 --mca mtl_base_verbose 10 ... in order to collect some logs. Bottom line, pml/cm should be selected (instead

[OMPI users] OpenMPI with PSM on True Scale with OmniPath drivers

2018-01-22 Thread William Hay
We have a couple of clusters with Qlogic Infinipath/Intel TrueScale networking. While testing a kernel upgrade we find that the Truescale drivers will no longer build against recent RHEL kernels. Intel tells us that the Omnipath drivers will work for True Scale adapters so we install those.