I am a new comer to Open MPI.
I have spent the last day trying to diagnose why a "hello world" MPI application compiled with OpenMPI v1.6.1 (64 bit) hangs when run on two EC2 Windows instances. I note they are running on different subnets so I'm using the mca btl_tcp_if_include 10.0.0.0/8 parameter. My two hosts are 10.242.73.81,10.116.114.238. I've placed the executable in the same path on both machines. Diagnostic info requested is attached along with sample application source. When I run two processes on one instance - the command succeeds: C:\mpi\exe>mpiexec -n 2 -host 10.242.73.81 --mca btl_tcp_if_include 10.0.0.0/8 MPIHello.exe WE have 2 processors Hello 1 Processor 1 at node AMAZONA-BMCKVD6 reporting for duty When I run across two hosts, the executable is launched on both instances but the process hangs: C:\mpi\exe>mpiexec -n 4 -host 10.242.73.81,10.116.114.238 --mca btl_tcp_if_include 10.0.0.0/8 MPIHello.exe connecting to 10.116.114.238 username:greenbutton password:********* Save Credential?(Y/N) n WE have 4 processors Re-running with debug: C:\mpi\exe>mpiexec -n 4 -host 10.242.73.81,10.116.114.238 -d --mca btl_tcp_if_include 10.0.0.0/8 MPIHello.exe [AMAZONA-BMCKVD6:01240] procdir: C:\Users\GREENB~1\AppData\Local\Temp\2\openmpi-sessions-greenbutton@AMAZONA- BMCKVD6_0\63746\0\0 [AMAZONA-BMCKVD6:01240] jobdir: C:\Users\GREENB~1\AppData\Local\Temp\2\openmpi-sessions-greenbutton@AMAZONA- BMCKVD6_0\63746\0 [AMAZONA-BMCKVD6:01240] top: openmpi-sessions-greenbutton@AMAZONA-BMCKVD6_0 [AMAZONA-BMCKVD6:01240] tmp: C:\Users\GREENB~1\AppData\Local\Temp\2 [AMAZONA-BMCKVD6:01240] mpiexec: reset PATH: C:\Program Files (x86)\OpenMPI_v1.6-x64\bin;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\Sy stem32\WindowsPowerShell\v1.0\; [AMAZONA-BMCKVD6:01240] mpiexec: reset LD_LIBRARY_PATH: C:\Program Files (x86)\OpenMPI_v1.6-x64\lib connecting to 10.116.114.238 username:greenbutton password:********* Save Credential?(Y/N) n [AMAZONA-BMCKVD6:02728] procdir: C:\Users\GREENB~1\AppData\Local\Temp\2\openmpi-sessions-greenbutton@AMAZONA- BMCKVD6_0\63746\1\0 [AMAZONA-BMCKVD6:02728] jobdir: C:\Users\GREENB~1\AppData\Local\Temp\2\openmpi-sessions-greenbutton@AMAZONA- BMCKVD6_0\63746\1 [AMAZONA-BMCKVD6:02728] top: openmpi-sessions-greenbutton@AMAZONA-BMCKVD6_0 [AMAZONA-BMCKVD6:02728] tmp: C:\Users\GREENB~1\AppData\Local\Temp\2 [AMAZONA-BMCKVD6:02728] [[63746,1],0] node[0].name AMAZONA-BMCKVD6 daemon 0 [AMAZONA-BMCKVD6:02728] [[63746,1],0] node[1].name 10 daemon 1 [AMAZONA-BMCKVD6:01500] procdir: C:\Users\GREENB~1\AppData\Local\Temp\2\openmpi-sessions-greenbutton@AMAZONA- BMCKVD6_0\63746\1\2 [AMAZONA-BMCKVD6:01500] jobdir: C:\Users\GREENB~1\AppData\Local\Temp\2\openmpi-sessions-greenbutton@AMAZONA- BMCKVD6_0\63746\1 [AMAZONA-BMCKVD6:01500] top: openmpi-sessions-greenbutton@AMAZONA-BMCKVD6_0 [AMAZONA-BMCKVD6:01500] tmp: C:\Users\GREENB~1\AppData\Local\Temp\2 [AMAZONA-BMCKVD6:01500] [[63746,1],2] node[0].name AMAZONA-BMCKVD6 daemon 0 [AMAZONA-BMCKVD6:01500] [[63746,1],2] node[1].name 10 daemon 1 WE have 4 processors I'd appreciate any guidance to getting this example to run on two instances on disparate subnets on Windows Server 2008 R2. Thanks in advance for your help. Regards, Peter Peter Soukalopoulos Development Team Leader | GreenButton Limited | <http://www.greenbutton.com/> www.greenbutton.com Level 13, Simpl House, 40 Mercer Street, Wellington, New Zealand Mobile: +64 22 632 5023| <mailto:peter.soukalopou...@greenbutton.com> peter.soukalopou...@greenbutton.com | Skype: psoukal | HQ: +644 499 0424 Description: Description: GreenButton_words_small Description: cid:image003.jpg@01CC4E01.BA075BC0 This message contains confidential information, intended only for the person(s) named above, which may also be privileged. Any use, distribution, copying or disclosure by any other person is strictly prohibited. In such case, you should delete this message and kindly notify the sender via reply e-mail. Please advise immediately if you or your employer does not consent to Internet e-mail for messages of this kind. ***************************************************************************** ** ** ** WARNING: This email contains an attachment of a very suspicious type. ** ** You are urged NOT to open this attachment unless you are absolutely ** ** sure it is legitimate. Opening this attachment may cause irreparable ** ** damage to your computer and your files. If you have any questions ** ** about the validity of this message, PLEASE SEEK HELP BEFORE OPENING IT. ** ** ** ** This warning was added by the IU Computer Science Dept. mail scanner. ** *****************************************************************************
<<attachment: MPIHello.zip>>
Windows IP Configuration Host Name . . . . . . . . . . . . : AMAZONA-BMCKVD6 Primary Dns Suffix . . . . . . . : Node Type . . . . . . . . . . . . : Hybrid IP Routing Enabled. . . . . . . . : No WINS Proxy Enabled. . . . . . . . : No DNS Suffix Search List. . . . . . : ec2.internal us-east-1.ec2-utilities.amazonaws.com compute-1.internal Ethernet adapter Local Area Connection 2: Connection-specific DNS Suffix . : ec2.internal Description . . . . . . . . . . . : RedHat PV NIC Driver #2 Physical Address. . . . . . . . . : 12-31-3B-01-46-A7 DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes Link-local IPv6 Address . . . . . : fe80::8de9:8318:dd91:7922%13(Preferred) IPv4 Address. . . . . . . . . . . : 10.242.73.81(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.255.0 Lease Obtained. . . . . . . . . . : Friday, June 22, 2012 2:29:09 AM Lease Expires . . . . . . . . . . : Saturday, June 23, 2012 2:29:10 AM Default Gateway . . . . . . . . . : 10.242.73.1 DHCP Server . . . . . . . . . . . : 169.254.1.0 DHCPv6 IAID . . . . . . . . . . . : 286404923 DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-17-50-59-19-12-31-39-03-B4-4B DNS Servers . . . . . . . . . . . : 172.16.0.23 NetBIOS over Tcpip. . . . . . . . : Enabled Tunnel adapter Local Area Connection* 11: Connection-specific DNS Suffix . : Description . . . . . . . . . . . : Teredo Tunneling Pseudo-Interface Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes IPv6 Address. . . . . . . . . . . : 2001:0:4137:9e76:3049:1854:f50d:b6ae(Preferred) Link-local IPv6 Address . . . . . : fe80::3049:1854:f50d:b6ae%12(Preferred) Default Gateway . . . . . . . . . : :: NetBIOS over Tcpip. . . . . . . . : Disabled Tunnel adapter isatap.ec2.internal: Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : ec2.internal Description . . . . . . . . . . . : Microsoft ISATAP Adapter #2 Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes
Windows IP Configuration Host Name . . . . . . . . . . . . : AMAZONA-BMCKVD6 Primary Dns Suffix . . . . . . . : Node Type . . . . . . . . . . . . : Hybrid IP Routing Enabled. . . . . . . . : No WINS Proxy Enabled. . . . . . . . : No DNS Suffix Search List. . . . . . : ec2.internal us-east-1.ec2-utilities.amazonaws.com compute-1.internal Ethernet adapter Local Area Connection 2: Connection-specific DNS Suffix . : ec2.internal Description . . . . . . . . . . . : RedHat PV NIC Driver #2 Physical Address. . . . . . . . . : 12-31-3D-02-65-20 DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes Link-local IPv6 Address . . . . . : fe80::48cc:d41b:6b46:656d%13(Preferred) IPv4 Address. . . . . . . . . . . : 10.116.114.238(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.254.0 Lease Obtained. . . . . . . . . . : Friday, June 22, 2012 2:29:10 AM Lease Expires . . . . . . . . . . : Saturday, June 23, 2012 2:29:11 AM Default Gateway . . . . . . . . . : 10.116.114.1 DHCP Server . . . . . . . . . . . : 169.254.1.0 DHCPv6 IAID . . . . . . . . . . . : 286404923 DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-17-50-59-19-12-31-39-03-B4-4B DNS Servers . . . . . . . . . . . : 172.16.0.23 NetBIOS over Tcpip. . . . . . . . : Enabled Tunnel adapter Local Area Connection* 11: Connection-specific DNS Suffix . : Description . . . . . . . . . . . : Teredo Tunneling Pseudo-Interface Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes IPv6 Address. . . . . . . . . . . : 2001:0:4137:9e76:3c31:383:f58b:8d11(Preferred) Link-local IPv6 Address . . . . . : fe80::3c31:383:f58b:8d11%12(Preferred) Default Gateway . . . . . . . . . : :: NetBIOS over Tcpip. . . . . . . . : Disabled Tunnel adapter isatap.ec2.internal: Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : ec2.internal Description . . . . . . . . . . . : Microsoft ISATAP Adapter #2 Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes
<<attachment: ompi_info.zip>>