Jon Mason wrote:
The new cpc selection framework is now in place. The patch below allows
for dynamic selection of cpc methods based on what is available. It
also allows for inclusion/exclusions of methods. It even futher allows
for modifying the priorities of certain cpc methods to better determine
the optimal cpc method.
This patch also contains XRC compile time disablement (per Jeff's
patch).
Need make sure that CM stuff will be disabled during compile tine if it
is not installed on machine.
At a high level, the cpc selections works by walking through each cpc
and allowing it to test to see if it is permissable to run on this
mpirun. It returns a priority if it is permissable or a -1 if not. All
of the cpc names and priorities are rolled into a string. This string
is then encapsulated in a message and passed around all the ompi
processes. Once received and unpacked, the list received is compared
to a local copy of the list. The connection method is chosen by
comparing the lists passed around to all nodes via modex with the list
generated locally. Any non-negative number is a potentially valid
connection method. The method below of determining the optimal
connection method is to take the cross-section of the two lists. The
highest single value (and the other side being non-negative) is selected
as the cpc method.
Please test it out. The tree can be found at
https://svn.open-mpi.org/svn/ompi/tmp-public/openib-cpc/
This patch has been tested with IB and iWARP adapters on a 2 node system
(with it correctly choosing to use oob and happily ignoring iWARP
adapters). It needs XRC testing and testing of larger node systems.
Did you run MTT over all thess changes ?
I have few machines with connectX and i will try to run MTT on Sunday.
--
Pavel Shamis (Pasha)
Mellanox Technologies
