Hello Ralph and Gilles
Thanks for the clarification. My understanding was that if a BTL was
specified to mpirun, then only BTL (and, therefore, the ob1 PML) will be
used. However, I always saw that is not the case and now I know why.
I do have PSM capable cards (Qlogic IB) in my nodes, and this t
my basic understanding is that ob1 works with btl, and cm works with mtl
(please someone corrects me if I am wrong)
an other way to put this is cm cannot use the tcp btl.
so I can only guess one mtl (PSM ?) is available, and so cm is preferred
over ob1.
what if you
mpirun --mca mtl ^psm ...
is cm
CM is not being selected for TCP - you specified TCP for the BTLs, but that
assumes that a BTL will be selected. You obviously have something in your
system that is supported by an MTL, and that will always be selected before a
BTL.
> On Apr 28, 2016, at 8:22 PM, dpchoudh . wrote:
>
> Hello
Hello Gilles
You are absolutely right:
1. Adding --mca pml_base_verbose 100 does show that it is the cm PML that
is being picked by default (even for TCP)
2. Adding --mca pml ob1 does cause add_procs() and related BTL friends to
be invoked.
With a command line of
mpirun -np 2 -hostfile ~/hostf
In Open MPI a process only retrieve information about a peer if they
communicate. Thus, the add_proc is called from the two sides of a
connection establishment, when locally a connection is decided or when a
network packet requires a the existence of a proc (for the initiator of the
connection). Th
the add_procs subroutine of the btl should be called.
/* i added a printf in mca_btl_tcp_add_procs and it *is* invoked */
can you try again with --mca pml ob1 --mca pml_base_verbose 100 ?
maybe the add_procs subroutine is not invoked because openmpi uses cm
instead of ob1
Cheers,
Gilles
