Hi I want to use SGE to run jobs on a cluster with mx and infiniband nodes. By dividing the nodes into two host groups SGE will submit to either interconnect.
The interconnect can be specified in the mpirun command with the --mca btl parameter. However users would then have to decide at runtime which of the two interconnects should be used, but if the job has to queue they just want the first available interconnect. I built OpenMPI 1.2.5 with both the mx and ofed options specified. If I understand some of the FAQs correctly then in that case the --mca parameter is not necessary and OpenMPI would find the correct interconnect because it is aware of both mx and ofed libraries. This does not seem to work, the job always ends up trying to use mx on openib nodes. Is there a way to inform mpirun, in cooperation with SGE, which btl should be used? I would be grateful for any advice Regards Henk
