Greetings,
Thanks for the clarity. I just had a few quick follow-up questions,
inline below.
brad
On Sat, 27 Aug 2005, Jeff Squyres wrote:
Particularly, I'm just curious about the difference between pml-bml-btl
and the old pml-ptl (as described in your TEG paper) and why changes
were made. I'm also trying to understand the differences between this
approach and LAM's RPI as well as the device-channel distinction made
in MPICH2; could anyone recommend something I could read or give me a
brief description here to satisfy my curiosities?
I can't speak for the MPICH2 device channel, but here's a summary of
the rest...
PML: Pretty much the same as it was described in the paper. Its
interface is basically MPI semantics (i.e., it sits right under
MPI_SEND and the rest).
BTL: Byte Transfer Layer; it's the next generation of PTL. The BTL is
much more simple than the PTL, and removes all vestigaes of any MPI
semantics that still lived in the PTL. It's a very simple byte mover
layer, intended to make it quite easy to implement new network
interfaces.
I was curious about what you meant by the removal of MPI semantics. Do
you mean it simply has no notion of tags, ranks, etc? In other words,
does it simply put the data into some sort of format so that the PML can
operate on with its own state machine?
Also, say you had some underlying protocol that allowed unordered delivery
of data (so not fully ordered like TCP); which "layer" would the notion of
"order" be handled in? I'm guessing PML would need some sort of sequence
number attached to it; is that right?
BML: BTL Management Layer; this used to be part of the PML but we
recently split it off into its own framework. It's mainly the utility
gorp of managing multiple BTL modules in a single process. This was
done because when working with the next generation of collectives,
MPI-2 IO, and MPI-2 one sided operations, we want to have the ability
to use the PML (which the collectives do today, for example) or to be
able to dive right down and directly use the BTLs (i.e., cut out a
little latency).
In the cases where the BML is required, does it cost extra memcpy's?
Also, has anyone ever compared the costs of OpenMPI striping using BMLs
versus a version where this is bypassed and something like channel bonding
is used? I'm just wondering how comparable they might be although I
understand that channel bonding might not be so portable...
LAM's RPI is actually sorta similar to the PML -- it implements the
MPI-level semantics. But nothing was broken out below the RPI -- it
had to implement the MPI-level semantics *and* do multi-device *and* be
the byte transfer layer.
I'm too used to this and trying to become familiar with the new way ;-)
One point worth clarifying -- the word "layer" here is somewhat
misleading. These are not so much "layers" (meaning that you *have* to
go through the PML and then through the BML and then through the BTL to
send bytes) as they are logical separation of functionality. There are
certainly cases where the BML is bypassed, and, as I mentioned above,
we want to investigate bypassing the PML as well.
So, guessing, is BML bypassed when only one device is available?
Does that help?
Indeed! Thanks again.
b
--
{+} Jeff Squyres
{+} The Open MPI Project
{+} http://www.open-mpi.org/
___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel
