The first approach sounds fair enough to me. We should avoid 2 and 3
as the pml selection mechanism used to be
more complex before we reduced it to accommodate a major design bug in
the BTL selection process. When using the complete PML selection, BTL
would be initialized several times, leading to a variety of bugs.
Eventually the PML selection should return to its old self, when the
BTL bug gets fixed.
Aurelien
Le 23 juin 08 à 12:36, Ralph H Castain a écrit :
Yo all
I've been doing further research into the modex and came across
something I
don't fully understand. It seems we have each process insert into
the modex
the name of the PML module that it selected. Once the modex has
exchanged
that info, it then loops across all procs in the job to check their
selection, and aborts if any proc picked a different PML module.
All well and good...assuming that procs actually -can- choose
different PML
modules and hence create an "abort" scenario. However, if I look
inside the
PML's at their selection logic, I find that a proc can ONLY pick a
module
other than ob1 if:
1. the user specifies the module to use via -mca pml xyz or by using a
module specific mca param to adjust its priority. In this case,
since the
mca param is propagated, ALL procs have no choice but to pick that
same
module, so that can't cause us to abort (we will have already
returned an
error and aborted if the specified module can't run).
2. the pml/cm module detects that an MTL module was selected, and
that it is
other than "psm". In this case, the CM module will be selected
because its
default priority is higher than that of OB1.
In looking deeper into the MTL selection logic, it appears to me
that you
either have the required capability or you don't. I can see that in
some
environments (e.g., rsh across unmanaged collections of machines),
it might
be possible for someone to launch across a set of machines where
some do and
some don't have the required support. However, in all other cases,
this will
be homogeneous across the system.
Given this analysis (and someone more familiar with the PML should
feel free
to confirm or correct it), it seems to me that this could be
streamlined via
one or more means:
1. at the most, we could have rank=0 add the PML module name to the
modex,
and other procs simply check it against their own and return an
error if
they differ. This accomplishes the identical functionality to what
we have
today, but with much less info in the modex.
2. we could eliminate this info from the modex altogether by
requiring the
user to specify the PML module if they want something other than the
default
OB1. In this case, there can be no confusion over what each proc is
to use.
The CM module will attempt to init the MTL - if it cannot do so,
then the
job will return the correct error and tell the user that CM/MTL
support is
unavailable.
3. we could again eliminate the info by not inserting it into the
modex if
(a) the default PML module is selected, or (b) the user specified
the PML
module to be used. In the first case, each proc can simply check to
see if
they picked the default - if not, then we can insert the info to
indicate
the difference. Thus, in the "standard" case, no info will be
inserted.
In the second case, we will already get an error if the specified
PML module
could not be used. Hence, the modex check provides no additional
info or
value.
I understand the motivation to support automation. However, in this
case,
the automation actually doesn't seem to buy us very much, and it isn't
coming "free". So perhaps some change in how this is done would be
in order?
Ralph
_______________________________________________
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel
