Hello,
Using today's SVN 1.4a1r19757
with
MCA='--mca btl_openib_verbose 1 --mca btl openib,self --mca btl_openib_if_include
"mlx4_0:1,mlx4_1:1"'
ibstatus (OFED 1.3.1) gives:
Infiniband device 'mlx4_0' port 1 status:
default gid: fe80:0000:0000:0000:0003:ba00:0100:71a1
base lid: 0x2f
sm lid: 0x1
state: 4: ACTIVE
phys state: 5: LinkUp
rate: 20 Gb/sec (4X DDR)
Infiniband device 'mlx4_0' port 2 status:
default gid: fe80:0000:0000:0000:0003:ba00:0100:71a2
base lid: 0x0
sm lid: 0x0
state: 1: DOWN
phys state: 2: Polling
rate: 10 Gb/sec (4X)
Infiniband device 'mlx4_1' port 1 status:
default gid: fe80:0000:0000:0000:0003:ba00:0100:70b9
base lid: 0x30
sm lid: 0x1
state: 4: ACTIVE
phys state: 5: LinkUp
rate: 20 Gb/sec (4X DDR)
Infiniband device 'mlx4_1' port 2 status:
default gid: fe80:0000:0000:0000:0003:ba00:0100:70ba
base lid: 0x0
sm lid: 0x0
state: 1: DOWN
phys state: 2: Polling
rate: 10 Gb/sec (4X)
OpenMPI says for a:
mpirun --prefix
/tools/openmpi/1.4a1r19757_svn/connectx/gcc64/4.1.2/openib/rh_EL_4/x86_64/xeon -x
LD_LIBRARY_PATH --mca btl_openib_verbose 1 --mca btl openib,self --mca
btl_openib_if_include "mlx4_0:1,mlx4_1:1" -np 4 -machinefile dhosts
./IMB-MPI1.openmpi
--------------------------------------------------------------------------
WARNING: One or more nonexistent OpenFabrics devices/ports were
specified:
Host: r4450_3
MCA parameter: mca_btl_if_include
Nonexistent entities: "mlx4_0:1,mlx4_1:1"
These entities will be ignored. You can disable this warning by
setting the btl_openib_warn_nonexistent_if MCA parameter to 0.
--------------------------------------------------------------------------
Scali 5.6 works in dual-rail mode with this as does mvapich2-1.2rc2.
What am I doing wrong, please?
DM
