On May 29, 2009, at 12:31 AM, shan axida wrote:
Is it true to use bidirectianal communication with MPI in ethernet
Cluster?
Are you asking if Open MPI uses bi-direction TCP sockets? Yes, it
does: we open one TCP socket between the MPI sender and receiver, and
if the order is reversed (receiver becomes sender), we'll use the same
socket.
I have tried once (I thought, it is possible because of fully duplex
swithes).
However, I could not get bandwidth improvement as I was expecting.
If you really are using hubs, then if you have processes A and B both
sending to each other simultaneously across the same link, you're
going to have contention and one of them will have to wait.
Even if you do have switches, there is a *wide* performance variation
of low-quality switches. Most low-cost ethernet 1GB switches perform
correctly, but do not necessarily provide the same high performance
that you can get with higher-cost switches (i.e., you get what you pay
for).
If you answer is YES, would you please tell me about pseudocode for
bidirectional communication ?
Thank you.
Axida
From: Jeff Squyres <jsquy...@cisco.com>
To: Open MPI Users <us...@open-mpi.org>
Sent: Wednesday, May 27, 2009 11:28:42 PM
Subject: Re: [OMPI users] How to use Multiple links with
OpenMPI??????????????????
Open MPI considers hosts differently than network links.
So you should only list the actual hostname in the hostfile, with
slots equal to the number of processors (4 in your case, I think?).
Once the MPI processes are launched, they each look around on the
host that they're running and find network paths to each of their
peers. If they are multiple paths between pairs of peers, Open MPI
will round-robin stripe messages across each of the links. We don't
really have an easy setting for each peer pair only using 1 link.
Indeed, since connectivity is bidirectional, the traffic patterns
become less obvious if you want MPI_COMM_WORLD rank X to only use
link Y -- what does that mean to the other 4 MPI processes on the
other host (with whom you have assumedly assigned their own
individual links as well)?
On May 26, 2009, at 12:24 AM, shan axida wrote:
> Hi everyone,
> I want to ask how to use multiple links (multiple NICs) with
OpenMPI.
> For example, how can I assign a link to each process, if there are
4 links
> and 4 processors on each node in our cluster?
> Is this a correct way?
> hostfile:
> ----------------------
> host1-eth0 slots=1
> host1-eth1 slots=1
> host1-eth2 slots=1
> host1-eth3 slots=1
> host2-eth0 slots=1
> host2-eth1 slots=1
> host2-eth2 slots=1
> host2-eth3 slots=1
> ... ...
> ... ...
> host16-eth0 slots=1
> host16-eth1 slots=1
> host16-eth2 slots=1
> host16-eth3 slots=1
> ------------------------
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
--Jeff Squyres
Cisco Systems
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
--
Jeff Squyres
Cisco Systems
