Re: [O-MPI devel] pml vs bml vs btl

[Jeff Squyres]

On Aug 26, 2005, at 5:52 PM, Brad Penoff wrote:

I was just curious about the distinction between the pml, bml, and btl
layers within OpenMPI and was wondering if there was any place (other 
than
the code ;-) where I could read about the design decisions.  Any 
pointers?

Unfortunately, not yet.

Particularly, I'm just curious about the difference between pml-bml-btl
and the old pml-ptl (as described in your TEG paper) and why changes
were made.  I'm also trying to understand the differences between this
approach and LAM's RPI as well as the device-channel distinction made 
in
MPICH2; could anyone recommend something I could read or give me a 
brief
description here to satisfy my curiosities?

I can't speak for the MPICH2 device channel, but here's a summary of 
the rest...

PML: Pretty much the same as it was described in the paper.  Its 
interface is basically MPI semantics (i.e., it sits right under 
MPI_SEND and the rest).

BTL: Byte Transfer Layer; it's the next generation of PTL.  The BTL is 
much more simple than the PTL, and removes all vestigaes of any MPI 
semantics that still lived in the PTL.  It's a very simple byte mover 
layer, intended to make it quite easy to implement new network 
interfaces.

BML: BTL Management Layer; this used to be part of the PML but we 
recently split it off into its own framework.  It's mainly the utility 
gorp of managing multiple BTL modules in a single process.  This was 
done because when working with the next generation of collectives, 
MPI-2 IO, and MPI-2 one sided operations, we want to have the ability 
to use the PML (which the collectives do today, for example) or to be 
able to dive right down and directly use the BTLs (i.e., cut out a 
little latency).

LAM's RPI is actually sorta similar to the PML -- it implements the 
MPI-level semantics.  But nothing was broken out below the RPI -- it 
had to implement the MPI-level semantics *and* do multi-device *and* be 
the byte transfer layer.

One point worth clarifying -- the word "layer" here is somewhat 
misleading.  These are not so much "layers" (meaning that you *have* to 
go through the PML and then through the BML and then through the BTL to 
send bytes) as they are logical separation of functionality.  There are 
certainly cases where the BML is bypassed, and, as I mentioned above, 
we want to investigate bypassing the PML as well.

Does that help?

--
{+} Jeff Squyres
{+} The Open MPI Project
{+} http://www.open-mpi.org/