Dear Open MPIers:
I have been using MPI for many years, most recently Open MPI. But I
have just encountered the first situation in which it will be helpful to
create communicators (for an unstructured sparse matrix algorithm).
I have identified two ways I could create the communicators I need.
Where P denotes the number of MPI processors, Option A is:
1. Exchange of messages between processors of adjacent rank
[O(1) message rounds (one up, one down)]
2. One scan operation
[O(log P) message rounds]
3. One or two calls to MPI_COMM_SPLIT
[Unknown complexity]
Option B is:
1. Three scan operations (one in reverse direction)
[O(log P) message rounds + time to make reverse communicator]
2. Each processor calls MPI_GROUP_RANGE_INCL and MPI_COMM_CREATE
at most twice
[Unknown complexity]
All the groups/communicators I am creating are stride-1 ranges of
contiguous processors from MPI_COMM_WORLD. Some of them could overlap
by one processor, hence the possible need to call MPI_COMM_SPLIT or
MPI_COMM_CREATE twice per processor.
Option A looks easier to code, but I wonder whether it will scale as
well, because I am not sure about the complexity of MPI_COMM_SPLIT.
What are the parallel message complexities of MPI_COMM_SPLIT and
MPI_COMM_CREATE? I poked around the web but could not find much on this
topic.
For option B, I will need to make a communicator that has the same
processes as MPI_COMM_WORLD, but in reverse order. This looks like it
can be done easily with MPI_GROUP_RANGE_INCL with a stride of -1, but
again I am not sure how much communication is required to set up the
communicator -- I would guess O(log P) rounds of messages.
Any advice or explanation you can offer would be much appreciated.
Professor Jonathan Eckstein
Rutgers University
