On Jun 30, 2010, at 12:36 AM, Paul Hilscher wrote:
> >Open MPI currently has very limited cartesian support -- it actually doesn't
> >remap anything.
>
> I see, OpenMPI doesn't remap anything; this explains probably why my runtime
> of my simulation varies sometimes between 30% for the same setup.
If you're relying on remapping and your environment would benefit from
remapping, then yes, that could be a factor here.
> >Would you have any interest in writing a partitioning algorithm for your
> >needs within the context of a plugin? I'd be happy to walk >you through the
> >process; it's not too complicated (although we should probably move the
> >discussion off to the Open MPI devel >mailing list).
>
> I guess after using for more than a decade Open Source Software, it's time to
> give something back :). ... so yes, I am willing to do that !!
Exxxcellent. :-)
> Because I am not yet experienced with OpenMPI internals, I would really
> appreciate your advice, if you could tell me where exactly I have to dig
> into.. I guess it should be around ompi_topo_create function, but how to
> write MPI_Cart_Create as a plugin, I will rely on you information. And do you
> know if MPICH, LAM etc. have an efficient implementation of MPI_Cart_Create ?
> so I can borrow some ideas from them....
Heh. I used to be the LAM/MPI guy; we stole some of LAM's topology stuff
(including the unity mapping) for Open MPI. :-)
What you really want to do is create a new "topo" plugin. Topo plugins are
automatically called behind the scenes for MPI_Cart_create and friends -- you
don't need to muck about in ompi_topo_create() at all.
If you look in ompi/mca/topo, that's the top-level directory for the topo
framework (meaning: all of our topo "glue" code is in there in the "base/"
subdirectory, and all topo plugins are subdirs in there as well). There's only
1 plugin at the moment: it's called "unity" -- because it doesn't do any
transforms.
Everything that the unity component does is contained in ompi/mca/topo/unity.
Before going any further, let's talk about some logistics.
1. Where to do this work? Possibilities are:
a. The v1.4 branch
b. The v1.5 branch
c. Our Subversion trunk
d. Elsewhere
a. is not a good choice; it's in it's "stable" phase, meaning that new features
are not supposed to be added, and the v1.5.0 release it potentially within the
next week or two. b. is also not a good choice; we tend to develop on the SVN
trunk and then migrate to the v1.5 branch as needed. c. is probably a
reasonable choice, but I'd actually prefer d; here's why...
In MPI-2.2, some new topology functions were added. I set about adding these
functions to Open MPI, which meant expanding the topo framework a bit (and the
unity plugin). I got about halfway through this work before being called off
to tend other fires. As with a lot of Open MPI development, I did this work in
a Mercurial branch off our subversion trunk.
Your offer inspires me to finally get off my butt and finish the MPI-2.2
topology functions in that Mercurial branch. Collaborating in the Mercurial
branch is probably a good place to do this work -- several of us OMPI
developers use bitbucket.org for Mercurial hosting, which offers free accounts
for collaborators:
http://bitbucket.org/jsquyres/ompi-topo-fixes
The above URL is where my Mercurial branch is for this work. If you sign up
for an account on bitbucket (it's free), I'll give you write access.
This work will eventually be committed back to the SVN trunk and then move in
to the v1.5 branch, meaning that it would be included in future v1.5.x releases.
2. For any contribution of "non-trivial" size, we need a signed copy of the
Open MPI 3rd party contribution agreement. I apologize about this, but it's
what we need to do to maintain cleanliness of the intellectual property in Open
MPI and guarantee that we can continue to distribute it under the BSD license.
See:
http://www.open-mpi.org/community/contribute/
Note that the form is pretty much the Apache contribution agreement; we
essentially s/Apache/Open MPI/g.
If it'll take a little time to get this done, that's no problem -- we can
collaborate in the bitbucket and just hold off bringing any of your work into
SVN until the agreement is signed.
3. In the bitbucket repo, I created an "example" topo plugin in addition to the
unity plugin. It'll never be used at run-time; it's sole purpose in life is to
be an example on how to write new topo plugins.
I'll need to re-evaluate the state of that repo and see where it is, what is
left to be done, etc. But I think it's at least in a compilable state and
should at least nominally work (the new MPI-2.2 stuff may not work yet).
4. Some references for you:
- Read https://svn.open-mpi.org/trac/ompi/browser/trunk/HACKING for how to
build OMPI from an SVN or Mercurial checkout.
- Read https://svn.open-mpi.org/trac/ompi/wiki/devel/CreateComponent for a
description of how to add a new plugin ("component") to Open MPI
- Read https://svn.open-mpi.org/trac/ompi/wiki/devel/Autogen to understand what
autogen.sh does in the build process.
--
Jeff Squyres
[email protected]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/
