Re: [OMPI devel] IPv6 support in OpenMPI?
On Mon, 3 Apr 2006, Christian Kauhaus wrote: 1. Put IPv[46] addresses into the machinefile. Since they are protocol-specific, both rsh/ssh uses them just the way they are. Yes, this would work. The machinefile could come from the user or from a batch/queueing system, but this should signify that the producer knows about the configuration of the machines and wants these specific addresses to be used. 2. Put hostnames into the machinefile. Both rsh/ssh and OpenMPI perform their own resolver lookup. From what I understand, OpenMPI wants to have its daemons running on the machines and then the interface configuration and communication channel establishment follows, so the 2 phases are not connected to each other. The question is then what to do about the names given in the machinefile ? Should they just be passed to the rsh/ssh client hoping that they would work ? Should there be an option for resolving the names to a user (or maybe admin) specific version of IP which should then be passed to rsh/ssh ? Or should there be a list of IP addresses that are tried in sequence ? Hmm, maybe all of the above, with some means to choose between them ? Your setup (IPv6 addresses given by the resolver, but no IPv6-aware ssh) could be handled in both ways: either by putting numeric IPv4-addresses into the machinefile, or by specifying 'ssh -4'. Yes. Please keep in mind however that the opposite situation might also occur (starting via IPv6, MPI communication via IPv4). -- Bogdan Costescu IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868 E-mail: bogdan.coste...@iwr.uni-heidelberg.de
[OMPI devel] IPv6 in OpenRTE
There has been a lot of discussion about IPv6 in Open MPI and OpenRTE recently. My comments here relate solely to OpenRTE and are intended to help provide some clarity to the discussion. OpenRTE communications are done via the Runtime Messaging Library (RML) API. The RML is really a strategy layer - it determines which transport will be used for the given message, and handles routing where required (e.g., between cells). In our component architecture, the RML is implemented as a framework - only one RML component can be selected and active in a process. Sitting under the RML is one or more transport systems - these are known as Out-Of-Band (OOB) components and reside in the oob framework. Because the OpenRTE messaging system must work in a heterogeneous environment, multiple OOB components can be selected and active at one time. The RML is responsible for picking the correct OOB to use to communicate to a specific process in the most efficient manner possible. Message destinations are specified in terms of OpenRTE process names - *not* IP addresses. Thus, a message is sent to a particular OpenRTE process name - it is the shared responsibility of the RML and its underlying OOB components to translate that into a network address. The exact role of the RML versus the OOB in that translation process has not yet been determined. Communication contact information for each process is provided to a process during startup in the form of URI's that contain the OpenRTE process name, IP address, and socket. A process is first given the URI for the head node process (HNP) of that cell. This is done so that the process can obtain subsequent information from the registry such as contact info for all other processes in the job, MPI-layer contact information, etc. The URI for each process clearly indicates whether IPv6 or IPv4 is to be used for contacting that process name. The system allows for multiple URI's to be provided for the same process name - selection of which one to use for a given message is done by the RML based on (a) interface availability (e.g., if only IPv4 is available, then that is the one used) and (b) network congestion. Hence, there is no ambiguity over which transport to use. In the case of IPv6 versus IPv4, the expectation was that there would be two OOB components, one each for these two protocols. The OOB components are selected based on local support - i.e., if the local system supports IPv6, then that component would be selected and available. Likewise, if the local system can support IPv4, that component would be selected too. I hope that helps clarify OpenRTE's operation. I truly believe that including IPv6 and IPv4 components in the OOB will be fairly simple to accomplish. Yes, there may be some duplicate code - if there is enough duplication, we can move the duplicate code into the OOB's base and let the two components share it. Otherwise, a little duplication isn't that big a deal. I'd be happy to answer further questions. I believe you will find that the Open MPI transport layer operates in a very similar manner, though I leave that to Tim and Galen to clarify. Ralph
Re: [OMPI devel] IPv6 support in OpenMPI?
Bogdan Costescu : >The question is then what to do about the names given in the >machinefile ? Should they just be passed to the rsh/ssh client hoping >that they would work ? Should there be an option for resolving the Just passing the contents of the machinefile is the way the current code works and I think this is the most sensible solution. Any situation where there the resolver returns IPv6 addresses, but the installed networking software like rsh/ssh is not able to cope with them looks clearly like an admin error to me. This is nothing that OpenMPI should try to fix. And remember that there is always the -4 switch in ssh. Christian -- Dipl.-Inf. Christian Kauhaus <>< Lehrstuhl fuer Rechnerarchitektur und -kommunikation Institut fuer Informatik * Ernst-Abbe-Platz 1-2 * D-07743 Jena Tel: +49 3641 9 46376 * Fax: +49 3641 9 46372 * Raum 3217
Re: [OMPI devel] IPv6 in OpenRTE
Ralph Castain : > There has been a lot of discussion about IPv6 in Open MPI and > OpenRTE recently. My comments here relate solely to OpenRTE and are > intended to help provide some clarity to the discussion. Thank you very much for your helpful clarifications. We are currently collecting some machines for a testing environment -- it does not make sense to develop something as multi-platform as OpenMPI/OpenRTE with only a bunch of x86 Linux boxen. We have quite a collection of different machines here at our institute, but nearly all of them have at least IPv6 headers available. What a pity. ;-) If someone could provide us an account on a non-IPv6-enabled Unix machine (like an old Solaris or HP-UX), this would be very helpful. This way we can be sure not to break the build with our patches. Christian -- Dipl.-Inf. Christian Kauhaus <>< Lehrstuhl fuer Rechnerarchitektur und -kommunikation Institut fuer Informatik * Ernst-Abbe-Platz 1-2 * D-07743 Jena Tel: +49 3641 9 46376 * Fax: +49 3641 9 46372 * Raum 3217
