Re: [OMPI users] choosing network: infiniband vs. ethernet

[Lana Deere via users]

I assume UCX is https://www.openucx.org?  (Google found several things
called UCX when I searched, but that seemed the right one.)  I will try
installing it and then reinstall OpenMPI.  Hopefully it will then choose
between network transports automatically based on what's available.  I'll
also look at the slides and see if I can make sense of them.  Thanks.

.. Lana (lana.de...@gmail.com)




On Sat, Jul 18, 2020 at 9:41 AM Jeff Squyres (jsquyres) 
wrote:

> On Jul 16, 2020, at 2:56 PM, Lana Deere via users <
> users@lists.open-mpi.org> wrote:
>
>
> I am new to open mpi.  I built 4.0.4 on a CentOS7 machine and tried doing
> an mpirun of a small program compiled against openmpi.  It seems to have
> failed because my host does not have infiniband.  I can't seem to figure
> out how I should configure when I build so it will do what I want, namely
> use infiniband if there are IB HCAs on the system and otherwise use the
> ethernet on the system.
>
>
> UCX is the underlying library that Mellanox/Nvidia prefers these days for
> use with MPI and InfiniBand.
>
> Meaning: you should first install UCX and then build Open MPI with
> --with-ucx=/directory/of/ucx/installation.
>
> We just hosted parts 1 and 2 of a seminar entitled "The ABCs of Open MPI"
> that covered topics like this.  Check out:
>
> https://www.open-mpi.org/video/?category=general#abcs-of-open-mpi-part-1
> and
> https://www.open-mpi.org/video/?category=general#abcs-of-open-mpi-part-2
>
> In particular, you might want to look at slides 28-42 in part 2 for a
> bunch of discussion about how Open MPI (by default) picks the underlying
> network / APIs to use, and then how you can override that if you want to.
>
> --
> Jeff Squyres
> jsquy...@cisco.com
>
>

Re: [OMPI users] choosing network: infiniband vs. ethernet

[Jeff Squyres (jsquyres) via users]

Correct, UCX = OpenUCX.org.

If you have the Mellanox drivers package installed, it probably would have 
installed UCX (and Open MPI).  You'll have to talk to your sysadmin and/or 
Mellanox support for details about that.


On Jul 20, 2020, at 11:36 AM, Lana Deere 
mailto:lana.de...@gmail.com>> wrote:

I assume UCX is https://www.openucx.org?  (Google 
found several things called UCX when I searched, but that seemed the right 
one.)  I will try installing it and then reinstall OpenMPI.  Hopefully it will 
then choose between network transports automatically based on what's available. 
 I'll also look at the slides and see if I can make sense of them.  Thanks.

.. Lana (lana.de...@gmail.com)




On Sat, Jul 18, 2020 at 9:41 AM Jeff Squyres (jsquyres) 
mailto:jsquy...@cisco.com>> wrote:
On Jul 16, 2020, at 2:56 PM, Lana Deere via users 
mailto:users@lists.open-mpi.org>> wrote:

I am new to open mpi.  I built 4.0.4 on a CentOS7 machine and tried doing an 
mpirun of a small program compiled against openmpi.  It seems to have failed 
because my host does not have infiniband.  I can't seem to figure out how I 
should configure when I build so it will do what I want, namely use infiniband 
if there are IB HCAs on the system and otherwise use the ethernet on the system.

UCX is the underlying library that Mellanox/Nvidia prefers these days for use 
with MPI and InfiniBand.

Meaning: you should first install UCX and then build Open MPI with 
--with-ucx=/directory/of/ucx/installation.

We just hosted parts 1 and 2 of a seminar entitled "The ABCs of Open MPI" that 
covered topics like this.  Check out:

https://www.open-mpi.org/video/?category=general#abcs-of-open-mpi-part-1
and
https://www.open-mpi.org/video/?category=general#abcs-of-open-mpi-part-2

In particular, you might want to look at slides 28-42 in part 2 for a bunch of 
discussion about how Open MPI (by default) picks the underlying network / APIs 
to use, and then how you can override that if you want to.

--
Jeff Squyres
jsquy...@cisco.com



--
Jeff Squyres
jsquy...@cisco.com

Re: [OMPI users] [External] Re: choosing network: infiniband vs. ethernet

[Prentice Bisbal via users]

Jeff,

Then you'll be happy to know I've been building OpenMPI for years and I 
never had any complaints about your configure/build system. Of course, 
I'm a pro who gets paid to build open-source software all day long, but 
I have to say I've never had any issues with configure, make, or 'make 
check' with any version of OpenMPI.


Keep up the great work!

Prentice

On 7/18/20 9:36 AM, Jeff Squyres (jsquyres) via users wrote:
Woo hoo!  I love getting emails like this.  We actually spend quite a 
bit of time in the design and implementation of the configure/build 
system so that it will "just work" in a wide variety of situations.


Thanks!


On Jul 17, 2020, at 5:43 PM, John Duffy via users 
mailto:users@lists.open-mpi.org>> wrote:


Hi Lana

I’m a Open MPI newbie too, but I managed to build Open MPI 4.0.4 
quite easily on Ubuntu 20.04 just following the instructions in 
README/INSTALL in the top level source directory, namely:


mkdir build
cd build
../configure CFLAGS=“-O3”  # My CFLAGS
make all
sudo make all
sudo make install

It just worked. My small cluster happily runs Open MPI over TCP/1GB 
Ethernet.


The make install step installed everything into /usr/local. I did 
forget to ldconfig initially, which confused me. Other than that it 
just worked.


John




--
Jeff Squyres
jsquy...@cisco.com 


--
Prentice Bisbal
Lead Software Engineer
Research Computing
Princeton Plasma Physics Laboratory
http://www.pppl.gov