Following is result of mpirun ompi_info on three_nodes.
three nodes version is same.
Package: Open MPI root@bwhead.clnet Distribution Open MPI root@bwslv01
Distribution Open MPI root@bwslv02 Distribution
Open MPI: 1.6.4 1.6.4 1.6.4
Open MPI SVN revision: r28081 r28081 r28081
Open MPI release date: Feb 19, 2013 Feb 19, 2013 Feb 19, 2013
Open RTE: 1.6.4 1.6.4 1.6.4
Open RTE SVN revision: r28081 r28081 r28081
Open RTE release date: Feb 19, 2013 Feb 19, 2013 Feb 19, 2013
OPAL: 1.6.4 1.6.4 1.6.4
OPAL SVN revision: r28081 r28081 r28081
OPAL release date: Feb 19, 2013 Feb 19, 2013 Feb 19, 2013
MPI API: 2.1 2.1 2.1
Ident string: 1.6.4 1.6.4 1.6.4
Prefix: /usr/local /usr/local /usr/local
Configured architecture: x86_64-unknown-linux-gnu
x86_64-unknown-linux-gnu x86_64-unknown-linux-gnu
Configure host: bwhead.clnet bwslv01 bwslv02
Configured by: root root root
Configured on: Wed May 8 20:38:14 JST 2013 45 JST 2013 29 JST 2013
Configure host: bwhead.clnet bwslv01 bwslv02
Built by: root root root
Built on: Wed May 8 20:48:44 JST 2013 43 JST 2013 38 JST 2013
Built host: bwhead.clnet bwslv01 bwslv02
C bindings: yes yes yes
C++ bindings: yes yes yes
Fortran77 bindings: yes (all) yes (all) yes (all)
Fortran90 bindings: yes yes yes
Fortran90 bindings size: small small small
C compiler: gcc gcc gcc
C compiler absolute: /usr/bin/gcc /usr/bin/gcc /usr/bin/gcc
C compiler family name: GNU GNU GNU
C compiler version: 4.4.7 4.4.7 4.4.7
C++ compiler: g++ g++ g++
C++ compiler absolute: /usr/bin/g++ /usr/bin/g++ /usr/bin/g++
Fortran77 compiler: gfortran gfortran gfortran
Fortran77 compiler abs: /usr/bin/gfortran /usr/bin/gfortran
/usr/bin/gfortran
Fortran90 compiler: gfortran gfortran gfortran
Fortran90 compiler abs: /usr/bin/gfortran /usr/bin/gfortran
/usr/bin/gfortran
C profiling: yes yes yes
C++ profiling: yes yes yes
Fortran77 profiling: yes yes yes
Fortran90 profiling: yes yes yes
C++ exceptions: no no no
Thread support: posix (MPI_THREAD_MULTIPLE: no, progress: no) no) no)
Sparse Groups: no no no
Internal debug support: no no no
MPI interface warnings: no no no
MPI parameter check: runtime runtime runtime
Memory profiling support: no no no
Memory debugging support: no no no
libltdl support: yes yes yes
Heterogeneous support: no no no
mpirun default --prefix: no no no
MPI I/O support: yes yes yes
MPI_WTIME support: gettimeofday gettimeofday gettimeofday
Symbol vis. support: yes yes yes
Host topology support: yes yes yes
MPI extensions: affinity example affinity example affinity example
FT Checkpoint support: no (checkpoint thread: no) no) no)
VampirTrace support: yes yes yes
MPI_MAX_PROCESSOR_NAME: 256 256 256
MPI_MAX_ERROR_STRING: 256 256 256
MPI_MAX_OBJECT_NAME: 64 64 64
MPI_MAX_INFO_KEY: 36 36 36
MPI_MAX_INFO_VAL: 256 256 256
MPI_MAX_PORT_NAME: 1024 1024 1024
MPI_MAX_DATAREP_STRING: 128 128 128
Package: Open MPI root@bwslv01 Distribution execinfo (MCA v2.0, API
v2.0, Component v1.6.4) execinfo (MCA v2.0, API v2.0, Component v1.6.4)
Open MPI: 1.6.4 linux (MCA v2.0, API v2.0, Component v1.6.4) linux (MCA
v2.0, API v2.0, Component v1.6.4)
Open MPI SVN revision: r28081 hwloc (MCA v2.0, API v2.0, Component
v1.6.4) hwloc (MCA v2.0, API v2.0, Component v1.6.4)
Open MPI release date: Feb 19, 2013 auto_detect (MCA v2.0, API v2.0,
Component v1.6.4) auto_detect (MCA v2.0, API v2.0, Component v1.6.4)
Open RTE: 1.6.4 file (MCA v2.0, API v2.0, Component v1.6.4) file (MCA
v2.0, API v2.0, Component v1.6.4)
Open RTE SVN revision: r28081 mmap (MCA v2.0, API v2.0, Component
v1.6.4) mmap (MCA v2.0, API v2.0, Component v1.6.4)
Open RTE release date: Feb 19, 2013 posix (MCA v2.0, API v2.0,
Component v1.6.4) posix (MCA v2.0, API v2.0, Component v1.6.4)
OPAL: 1.6.4 sysv (MCA v2.0, API v2.0, Component v1.6.4) sysv (MCA
v2.0, API v2.0, Component v1.6.4)
OPAL SVN revision: r28081 first_use (MCA v2.0, API v2.0, Component
v1.6.4) first_use (MCA v2.0, API v2.0, Component v1.6.4)
OPAL release date: Feb 19, 2013 hwloc (MCA v2.0, API v2.0, Component
v1.6.4) hwloc (MCA v2.0, API v2.0, Component v1.6.4)
MPI API: 2.1 linux (MCA v2.0, API v2.0, Component v1.6.4) linux (MCA
v2.0, API v2.0, Component v1.6.4)
Ident string: 1.6.4 env (MCA v2.0, API v2.0, Component v1.6.4) env (MCA
v2.0, API v2.0, Component v1.6.4)
Prefix: /usr/local config (MCA v2.0, API v2.0, Component v1.6.4) config
(MCA v2.0, API v2.0, Component v1.6.4)
Configured architecture: x86_64-unknown-linux-gnu linux (MCA v2.0, API
v2.0, Component v1.6.4) linux (MCA v2.0, API v2.0, Component v1.6.4)
Configure host: bwslv01 hwloc132 (MCA v2.0, API v2.0, Component
v1.6.4) hwloc132 (MCA v2.0, API v2.0, Component v1.6.4)
Configured by: root orte (MCA v2.0, API v2.0, Component v1.6.4) orte
(MCA v2.0, API v2.0, Component v1.6.4)
Configured on: Wed May 8 20:56:45 JST 2013 orte (MCA v2.0, API v2.0,
Component v1.6.4) orte (MCA v2.0, API v2.0, Component v1.6.4)
Configure host: bwslv01 basic (MCA v2.0, API v2.0, Component v1.6.4)
basic (MCA v2.0, API v2.0, Component v1.6.4)
Built by: root bucket (MCA v2.0, API v2.0, Component v1.6.4) bucket
(MCA v2.0, API v2.0, Component v1.6.4)
Built on: Wed May 8 21:05:43 JST 2013 basic (MCA v2.0, API v2.0,
Component v1.6.4) basic (MCA v2.0, API v2.0, Component v1.6.4)
Built host: bwslv01 hierarch (MCA v2.0, API v2.0, Component v1.6.4)
hierarch (MCA v2.0, API v2.0, Component v1.6.4)
C bindings: yes inter (MCA v2.0, API v2.0, Component v1.6.4) inter (MCA
v2.0, API v2.0, Component v1.6.4)
C++ bindings: yes self (MCA v2.0, API v2.0, Component v1.6.4) self (MCA
v2.0, API v2.0, Component v1.6.4)
Fortran77 bindings: yes (all) sm (MCA v2.0, API v2.0, Component
v1.6.4) sm (MCA v2.0, API v2.0, Component v1.6.4)
Fortran90 bindings: yes sync (MCA v2.0, API v2.0, Component v1.6.4)
sync (MCA v2.0, API v2.0, Component v1.6.4)
Fortran90 bindings size: small tuned (MCA v2.0, API v2.0, Component
v1.6.4) tuned (MCA v2.0, API v2.0, Component v1.6.4)
C compiler: gcc romio (MCA v2.0, API v2.0, Component v1.6.4) romio (MCA
v2.0, API v2.0, Component v1.6.4)
C compiler absolute: /usr/bin/gcc fake (MCA v2.0, API v2.0, Component
v1.6.4) fake (MCA v2.0, API v2.0, Component v1.6.4)
C compiler family name: GNU rdma (MCA v2.0, API v2.0, Component
v1.6.4) rdma (MCA v2.0, API v2.0, Component v1.6.4)
C compiler version: 4.4.7 sm (MCA v2.0, API v2.0, Component v1.6.4) sm
(MCA v2.0, API v2.0, Component v1.6.4)
C++ compiler: g++ bfo (MCA v2.0, API v2.0, Component v1.6.4) bfo (MCA
v2.0, API v2.0, Component v1.6.4)
C++ compiler absolute: /usr/bin/g++ csum (MCA v2.0, API v2.0, Component
v1.6.4) csum (MCA v2.0, API v2.0, Component v1.6.4)
Fortran77 compiler: gfortran ob1 (MCA v2.0, API v2.0, Component
v1.6.4) ob1 (MCA v2.0, API v2.0, Component v1.6.4)
Fortran77 compiler abs: /usr/bin/gfortran v (MCA v2.0, API v2.0,
Component v1.6.4) v (MCA v2.0, API v2.0, Component v1.6.4)
Fortran90 compiler: gfortran r2 (MCA v2.0, API v2.0, Component v1.6.4)
r2 (MCA v2.0, API v2.0, Component v1.6.4)
Fortran90 compiler abs: /usr/bin/gfortran vma (MCA v2.0, API v2.0,
Component v1.6.4) vma (MCA v2.0, API v2.0, Component v1.6.4)
C profiling: yes self (MCA v2.0, API v2.0, Component v1.6.4) self (MCA
v2.0, API v2.0, Component v1.6.4)
C++ profiling: yes sm (MCA v2.0, API v2.0, Component v1.6.4) sm (MCA
v2.0, API v2.0, Component v1.6.4)
Fortran77 profiling: yes tcp (MCA v2.0, API v2.0, Component v1.6.4)
tcp (MCA v2.0, API v2.0, Component v1.6.4)
Fortran90 profiling: yes unity (MCA v2.0, API v2.0, Component v1.6.4)
unity (MCA v2.0, API v2.0, Component v1.6.4)
C++ exceptions: no pt2pt (MCA v2.0, API v2.0, Component v1.6.4) pt2pt
(MCA v2.0, API v2.0, Component v1.6.4)
Thread support: posix (MPI_THREAD_MULTIPLE: no, progress: no) rdma (MCA
v2.0, API v2.0, Component v1.6.4) rdma (MCA v2.0, API v2.0, Component
v1.6.4)
Sparse Groups: no hnp (MCA v2.0, API v2.0, Component v1.6.4) hnp (MCA
v2.0, API v2.0, Component v1.6.4)
Internal debug support: no orted (MCA v2.0, API v2.0, Component
v1.6.4) orted (MCA v2.0, API v2.0, Component v1.6.4)
MPI interface warnings: no tool (MCA v2.0, API v2.0, Component v1.6.4)
tool (MCA v2.0, API v2.0, Component v1.6.4)
MPI parameter check: runtime tcp (MCA v2.0, API v2.0, Component
v1.6.4) tcp (MCA v2.0, API v2.0, Component v1.6.4)
Memory profiling support: no default (MCA v2.0, API v2.0, Component
v1.6.4) default (MCA v2.0, API v2.0, Component v1.6.4)
Memory debugging support: no cm (MCA v2.0, API v2.0, Component v1.6.4)
cm (MCA v2.0, API v2.0, Component v1.6.4)
libltdl support: yes loadleveler (MCA v2.0, API v2.0, Component
v1.6.4) loadleveler (MCA v2.0, API v2.0, Component v1.6.4)
Heterogeneous support: no slurm (MCA v2.0, API v2.0, Component v1.6.4)
slurm (MCA v2.0, API v2.0, Component v1.6.4)
mpirun default --prefix: no load_balance (MCA v2.0, API v2.0, Component
v1.6.4) load_balance (MCA v2.0, API v2.0, Component v1.6.4)
MPI I/O support: yes rank_file (MCA v2.0, API v2.0, Component v1.6.4)
rank_file (MCA v2.0, API v2.0, Component v1.6.4)
MPI_WTIME support: gettimeofday resilient (MCA v2.0, API v2.0,
Component v1.6.4) resilient (MCA v2.0, API v2.0, Component v1.6.4)
Symbol vis. support: yes round_robin (MCA v2.0, API v2.0, Component
v1.6.4) round_robin (MCA v2.0, API v2.0, Component v1.6.4)
Host topology support: yes seq (MCA v2.0, API v2.0, Component v1.6.4)
seq (MCA v2.0, API v2.0, Component v1.6.4)
MPI extensions: affinity example topo (MCA v2.0, API v2.0, Component
v1.6.4) topo (MCA v2.0, API v2.0, Component v1.6.4)
FT Checkpoint support: no (checkpoint thread: no) oob (MCA v2.0, API
v2.0, Component v1.6.4) oob (MCA v2.0, API v2.0, Component v1.6.4)
VampirTrace support: yes binomial (MCA v2.0, API v2.0, Component
v1.6.4) binomial (MCA v2.0, API v2.0, Component v1.6.4)
MPI_MAX_PROCESSOR_NAME: 256 cm (MCA v2.0, API v2.0, Component v1.6.4)
cm (MCA v2.0, API v2.0, Component v1.6.4)
MPI_MAX_ERROR_STRING: 256 direct (MCA v2.0, API v2.0, Component
v1.6.4) direct (MCA v2.0, API v2.0, Component v1.6.4)
MPI_MAX_OBJECT_NAME: 64 linear (MCA v2.0, API v2.0, Component v1.6.4)
linear (MCA v2.0, API v2.0, Component v1.6.4)
MPI_MAX_INFO_KEY: 36 radix (MCA v2.0, API v2.0, Component v1.6.4)
radix (MCA v2.0, API v2.0, Component v1.6.4)
MPI_MAX_INFO_VAL: 256 slave (MCA v2.0, API v2.0, Component v1.6.4)
slave (MCA v2.0, API v2.0, Component v1.6.4)
MPI_MAX_PORT_NAME: 1024 rsh (MCA v2.0, API v2.0, Component v1.6.4) rsh
(MCA v2.0, API v2.0, Component v1.6.4)
MPI_MAX_DATAREP_STRING: 128 slurm (MCA v2.0, API v2.0, Component
v1.6.4) slurm (MCA v2.0, API v2.0, Component v1.6.4)
Package: Open MPI root@bwslv02 Distribution rsh (MCA v2.0, API v2.0,
Component v1.6.4) rsh (MCA v2.0, API v2.0, Component v1.6.4)
Open MPI: 1.6.4 default (MCA v2.0, API v2.0, Component v1.6.4) default
(MCA v2.0, API v2.0, Component v1.6.4)
Open MPI SVN revision: r28081 env (MCA v2.0, API v2.0, Component
v1.6.4) env (MCA v2.0, API v2.0, Component v1.6.4)
Open MPI release date: Feb 19, 2013 hnp (MCA v2.0, API v2.0, Component
v1.6.4) hnp (MCA v2.0, API v2.0, Component v1.6.4)
Open RTE: 1.6.4 singleton (MCA v2.0, API v2.0, Component v1.6.4)
singleton (MCA v2.0, API v2.0, Component v1.6.4)
Open RTE SVN revision: r28081 slave (MCA v2.0, API v2.0, Component
v1.6.4) slave (MCA v2.0, API v2.0, Component v1.6.4)
Open RTE release date: Feb 19, 2013 slurm (MCA v2.0, API v2.0,
Component v1.6.4) slurm (MCA v2.0, API v2.0, Component v1.6.4)
OPAL: 1.6.4 slurmd (MCA v2.0, API v2.0, Component v1.6.4) slurmd (MCA
v2.0, API v2.0, Component v1.6.4)
OPAL SVN revision: r28081 tool (MCA v2.0, API v2.0, Component v1.6.4)
tool (MCA v2.0, API v2.0, Component v1.6.4)
OPAL release date: Feb 19, 2013 bad (MCA v2.0, API v2.0, Component
v1.6.4) bad (MCA v2.0, API v2.0, Component v1.6.4)
MPI API: 2.1 basic (MCA v2.0, API v2.0, Component v1.6.4) basic (MCA
v2.0, API v2.0, Component v1.6.4)
Ident string: 1.6.4 hier (MCA v2.0, API v2.0, Component v1.6.4) hier
(MCA v2.0, API v2.0, Component v1.6.4)
Prefix: /usr/local command (MCA v2.0, API v1.0, Component v1.6.4)
command (MCA v2.0, API v1.0, Component v1.6.4)
Configured architecture: x86_64-unknown-linux-gnu syslog (MCA v2.0, API
v1.0, Component v1.6.4) syslog (MCA v2.0, API v1.0, Component v1.6.4)
Configure host: bwslv02
Configured by: root
Configured on: Wed May 8 20:56:29 JST 2013
Configure host: bwslv02
Built by: root
Built on: Wed May 8 21:05:38 JST 2013
Built host: bwslv02
C bindings: yes
C++ bindings: yes
Fortran77 bindings: yes (all)
Fortran90 bindings: yes
Fortran90 bindings size: small
C compiler: gcc
C compiler absolute: /usr/bin/gcc
C compiler family name: GNU
C compiler version: 4.4.7
C++ compiler: g++
C++ compiler absolute: /usr/bin/g++
Fortran77 compiler: gfortran
Fortran77 compiler abs: /usr/bin/gfortran
Fortran90 compiler: gfortran
Fortran90 compiler abs: /usr/bin/gfortran
C profiling: yes
C++ profiling: yes
Fortran77 profiling: yes
Fortran90 profiling: yes
C++ exceptions: no
Thread support: posix (MPI_THREAD_MULTIPLE: no, progress: no)
Sparse Groups: no
Internal debug support: no
MPI interface warnings: no
MPI parameter check: runtime
Memory profiling support: no
Memory debugging support: no
libltdl support: yes
Heterogeneous support: no
mpirun default --prefix: no
MPI I/O support: yes
MPI_WTIME support: gettimeofday
Symbol vis. support: yes
Host topology support: yes
MPI extensions: affinity example
FT Checkpoint support: no (checkpoint thread: no)
VampirTrace support: yes
MPI_MAX_PROCESSOR_NAME: 256
MPI_MAX_ERROR_STRING: 256
MPI_MAX_OBJECT_NAME: 64
MPI_MAX_INFO_KEY: 36
MPI_MAX_INFO_VAL: 256
MPI_MAX_PORT_NAME: 1024
MPI_MAX_DATAREP_STRING: 128
MCA backtrace: execinfo (MCA v2.0, API v2.0, Component v1.6.4)
MCA memory: linux (MCA v2.0, API v2.0, Component v1.6.4)
MCA paffinity: hwloc (MCA v2.0, API v2.0, Component v1.6.4)
MCA carto: auto_detect (MCA v2.0, API v2.0, Component v1.6.4)
MCA carto: file (MCA v2.0, API v2.0, Component v1.6.4)
MCA shmem: mmap (MCA v2.0, API v2.0, Component v1.6.4)
MCA shmem: posix (MCA v2.0, API v2.0, Component v1.6.4)
MCA shmem: sysv (MCA v2.0, API v2.0, Component v1.6.4)
MCA maffinity: first_use (MCA v2.0, API v2.0, Component v1.6.4)
MCA maffinity: hwloc (MCA v2.0, API v2.0, Component v1.6.4)
MCA timer: linux (MCA v2.0, API v2.0, Component v1.6.4)
MCA installdirs: env (MCA v2.0, API v2.0, Component v1.6.4)
MCA installdirs: config (MCA v2.0, API v2.0, Component v1.6.4)
MCA sysinfo: linux (MCA v2.0, API v2.0, Component v1.6.4)
MCA hwloc: hwloc132 (MCA v2.0, API v2.0, Component v1.6.4)
MCA dpm: orte (MCA v2.0, API v2.0, Component v1.6.4)
MCA pubsub: orte (MCA v2.0, API v2.0, Component v1.6.4)
MCA allocator: basic (MCA v2.0, API v2.0, Component v1.6.4)
MCA allocator: bucket (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: basic (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: hierarch (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: inter (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: self (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: sm (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: sync (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: tuned (MCA v2.0, API v2.0, Component v1.6.4)
MCA io: romio (MCA v2.0, API v2.0, Component v1.6.4)
MCA mpool: fake (MCA v2.0, API v2.0, Component v1.6.4)
MCA mpool: rdma (MCA v2.0, API v2.0, Component v1.6.4)
MCA mpool: sm (MCA v2.0, API v2.0, Component v1.6.4)
MCA pml: bfo (MCA v2.0, API v2.0, Component v1.6.4)
MCA pml: csum (MCA v2.0, API v2.0, Component v1.6.4)
MCA pml: ob1 (MCA v2.0, API v2.0, Component v1.6.4)
MCA pml: v (MCA v2.0, API v2.0, Component v1.6.4)
MCA bml: r2 (MCA v2.0, API v2.0, Component v1.6.4)
MCA rcache: vma (MCA v2.0, API v2.0, Component v1.6.4)
MCA btl: self (MCA v2.0, API v2.0, Component v1.6.4)
MCA btl: sm (MCA v2.0, API v2.0, Component v1.6.4)
MCA btl: tcp (MCA v2.0, API v2.0, Component v1.6.4)
MCA topo: unity (MCA v2.0, API v2.0, Component v1.6.4)
MCA osc: pt2pt (MCA v2.0, API v2.0, Component v1.6.4)
MCA osc: rdma (MCA v2.0, API v2.0, Component v1.6.4)
MCA iof: hnp (MCA v2.0, API v2.0, Component v1.6.4)
MCA iof: orted (MCA v2.0, API v2.0, Component v1.6.4)
MCA iof: tool (MCA v2.0, API v2.0, Component v1.6.4)
MCA oob: tcp (MCA v2.0, API v2.0, Component v1.6.4)
MCA odls: default (MCA v2.0, API v2.0, Component v1.6.4)
MCA ras: cm (MCA v2.0, API v2.0, Component v1.6.4)
MCA ras: loadleveler (MCA v2.0, API v2.0, Component v1.6.4)
MCA ras: slurm (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: load_balance (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: rank_file (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: resilient (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: round_robin (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: seq (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: topo (MCA v2.0, API v2.0, Component v1.6.4)
MCA rml: oob (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: binomial (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: cm (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: direct (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: linear (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: radix (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: slave (MCA v2.0, API v2.0, Component v1.6.4)
MCA plm: rsh (MCA v2.0, API v2.0, Component v1.6.4)
MCA plm: slurm (MCA v2.0, API v2.0, Component v1.6.4)
MCA filem: rsh (MCA v2.0, API v2.0, Component v1.6.4)
MCA errmgr: default (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: env (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: hnp (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: singleton (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: slave (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: slurm (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: slurmd (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: tool (MCA v2.0, API v2.0, Component v1.6.4)
MCA grpcomm: bad (MCA v2.0, API v2.0, Component v1.6.4)
MCA grpcomm: basic (MCA v2.0, API v2.0, Component v1.6.4)
MCA grpcomm: hier (MCA v2.0, API v2.0, Component v1.6.4)
MCA notifier: command (MCA v2.0, API v1.0, Component v1.6.4)
MCA notifier: syslog (MCA v2.0, API v1.0, Component v1.6.4)
MCA backtrace: execinfo (MCA v2.0, API v2.0, Component v1.6.4)
MCA memory: linux (MCA v2.0, API v2.0, Component v1.6.4)
MCA paffinity: hwloc (MCA v2.0, API v2.0, Component v1.6.4)
MCA carto: auto_detect (MCA v2.0, API v2.0, Component v1.6.4)
MCA carto: file (MCA v2.0, API v2.0, Component v1.6.4)
MCA shmem: mmap (MCA v2.0, API v2.0, Component v1.6.4)
MCA shmem: posix (MCA v2.0, API v2.0, Component v1.6.4)
MCA shmem: sysv (MCA v2.0, API v2.0, Component v1.6.4)
MCA maffinity: first_use (MCA v2.0, API v2.0, Component v1.6.4)
MCA maffinity: hwloc (MCA v2.0, API v2.0, Component v1.6.4)
MCA timer: linux (MCA v2.0, API v2.0, Component v1.6.4)
MCA installdirs: env (MCA v2.0, API v2.0, Component v1.6.4)
MCA installdirs: config (MCA v2.0, API v2.0, Component v1.6.4)
MCA sysinfo: linux (MCA v2.0, API v2.0, Component v1.6.4)
MCA hwloc: hwloc132 (MCA v2.0, API v2.0, Component v1.6.4)
MCA dpm: orte (MCA v2.0, API v2.0, Component v1.6.4)
MCA pubsub: orte (MCA v2.0, API v2.0, Component v1.6.4)
MCA allocator: basic (MCA v2.0, API v2.0, Component v1.6.4)
MCA allocator: bucket (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: basic (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: hierarch (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: inter (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: self (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: sm (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: sync (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: tuned (MCA v2.0, API v2.0, Component v1.6.4)
MCA io: romio (MCA v2.0, API v2.0, Component v1.6.4)
MCA mpool: fake (MCA v2.0, API v2.0, Component v1.6.4)
MCA mpool: rdma (MCA v2.0, API v2.0, Component v1.6.4)
MCA mpool: sm (MCA v2.0, API v2.0, Component v1.6.4)
MCA pml: bfo (MCA v2.0, API v2.0, Component v1.6.4)
MCA pml: csum (MCA v2.0, API v2.0, Component v1.6.4)
MCA pml: ob1 (MCA v2.0, API v2.0, Component v1.6.4)
MCA pml: v (MCA v2.0, API v2.0, Component v1.6.4)
MCA bml: r2 (MCA v2.0, API v2.0, Component v1.6.4)
MCA rcache: vma (MCA v2.0, API v2.0, Component v1.6.4)
MCA btl: self (MCA v2.0, API v2.0, Component v1.6.4)
MCA btl: sm (MCA v2.0, API v2.0, Component v1.6.4)
MCA btl: tcp (MCA v2.0, API v2.0, Component v1.6.4)
MCA topo: unity (MCA v2.0, API v2.0, Component v1.6.4)
MCA osc: pt2pt (MCA v2.0, API v2.0, Component v1.6.4)
MCA osc: rdma (MCA v2.0, API v2.0, Component v1.6.4)
MCA iof: hnp (MCA v2.0, API v2.0, Component v1.6.4)
MCA iof: orted (MCA v2.0, API v2.0, Component v1.6.4)
MCA iof: tool (MCA v2.0, API v2.0, Component v1.6.4)
MCA oob: tcp (MCA v2.0, API v2.0, Component v1.6.4)
MCA odls: default (MCA v2.0, API v2.0, Component v1.6.4)
MCA ras: cm (MCA v2.0, API v2.0, Component v1.6.4)
MCA ras: loadleveler (MCA v2.0, API v2.0, Component v1.6.4)
MCA ras: slurm (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: load_balance (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: rank_file (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: resilient (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: round_robin (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: seq (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: topo (MCA v2.0, API v2.0, Component v1.6.4)
MCA rml: oob (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: binomial (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: cm (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: direct (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: linear (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: radix (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: slave (MCA v2.0, API v2.0, Component v1.6.4)
MCA plm: rsh (MCA v2.0, API v2.0, Component v1.6.4)
MCA plm: slurm (MCA v2.0, API v2.0, Component v1.6.4)
MCA filem: rsh (MCA v2.0, API v2.0, Component v1.6.4)
MCA errmgr: default (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: env (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: hnp (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: singleton (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: slave (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: slurm (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: slurmd (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: tool (MCA v2.0, API v2.0, Component v1.6.4)
MCA grpcomm: bad (MCA v2.0, API v2.0, Component v1.6.4)
MCA grpcomm: basic (MCA v2.0, API v2.0, Component v1.6.4)
MCA grpcomm: hier (MCA v2.0, API v2.0, Component v1.6.4)
MCA notifier: command (MCA v2.0, API v1.0, Component v1.6.4)
MCA notifier: syslog (MCA v2.0, API v1.0, Component v1.6.4)
MCA backtrace: execinfo (MCA v2.0, API v2.0, Component v1.6.4)
MCA memory: linux (MCA v2.0, API v2.0, Component v1.6.4)
MCA paffinity: hwloc (MCA v2.0, API v2.0, Component v1.6.4)
MCA carto: auto_detect (MCA v2.0, API v2.0, Component v1.6.4)
MCA carto: file (MCA v2.0, API v2.0, Component v1.6.4)
MCA shmem: mmap (MCA v2.0, API v2.0, Component v1.6.4)
MCA shmem: posix (MCA v2.0, API v2.0, Component v1.6.4)
MCA shmem: sysv (MCA v2.0, API v2.0, Component v1.6.4)
MCA maffinity: first_use (MCA v2.0, API v2.0, Component v1.6.4)
MCA maffinity: hwloc (MCA v2.0, API v2.0, Component v1.6.4)
MCA timer: linux (MCA v2.0, API v2.0, Component v1.6.4)
MCA installdirs: env (MCA v2.0, API v2.0, Component v1.6.4)
MCA installdirs: config (MCA v2.0, API v2.0, Component v1.6.4)
MCA sysinfo: linux (MCA v2.0, API v2.0, Component v1.6.4)
MCA hwloc: hwloc132 (MCA v2.0, API v2.0, Component v1.6.4)
MCA dpm: orte (MCA v2.0, API v2.0, Component v1.6.4)
MCA pubsub: orte (MCA v2.0, API v2.0, Component v1.6.4)
MCA allocator: basic (MCA v2.0, API v2.0, Component v1.6.4)
MCA allocator: bucket (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: basic (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: hierarch (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: inter (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: self (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: sm (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: sync (MCA v2.0, API v2.0, Component v1.6.4)
MCA coll: tuned (MCA v2.0, API v2.0, Component v1.6.4)
MCA io: romio (MCA v2.0, API v2.0, Component v1.6.4)
MCA mpool: fake (MCA v2.0, API v2.0, Component v1.6.4)
MCA mpool: rdma (MCA v2.0, API v2.0, Component v1.6.4)
MCA mpool: sm (MCA v2.0, API v2.0, Component v1.6.4)
MCA pml: bfo (MCA v2.0, API v2.0, Component v1.6.4)
MCA pml: csum (MCA v2.0, API v2.0, Component v1.6.4)
MCA pml: ob1 (MCA v2.0, API v2.0, Component v1.6.4)
MCA pml: v (MCA v2.0, API v2.0, Component v1.6.4)
MCA bml: r2 (MCA v2.0, API v2.0, Component v1.6.4)
MCA rcache: vma (MCA v2.0, API v2.0, Component v1.6.4)
MCA btl: self (MCA v2.0, API v2.0, Component v1.6.4)
MCA btl: sm (MCA v2.0, API v2.0, Component v1.6.4)
MCA btl: tcp (MCA v2.0, API v2.0, Component v1.6.4)
MCA topo: unity (MCA v2.0, API v2.0, Component v1.6.4)
MCA osc: pt2pt (MCA v2.0, API v2.0, Component v1.6.4)
MCA osc: rdma (MCA v2.0, API v2.0, Component v1.6.4)
MCA iof: hnp (MCA v2.0, API v2.0, Component v1.6.4)
MCA iof: orted (MCA v2.0, API v2.0, Component v1.6.4)
MCA iof: tool (MCA v2.0, API v2.0, Component v1.6.4)
MCA oob: tcp (MCA v2.0, API v2.0, Component v1.6.4)
MCA odls: default (MCA v2.0, API v2.0, Component v1.6.4)
MCA ras: cm (MCA v2.0, API v2.0, Component v1.6.4)
MCA ras: loadleveler (MCA v2.0, API v2.0, Component v1.6.4)
MCA ras: slurm (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: load_balance (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: rank_file (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: resilient (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: round_robin (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: seq (MCA v2.0, API v2.0, Component v1.6.4)
MCA rmaps: topo (MCA v2.0, API v2.0, Component v1.6.4)
MCA rml: oob (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: binomial (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: cm (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: direct (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: linear (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: radix (MCA v2.0, API v2.0, Component v1.6.4)
MCA routed: slave (MCA v2.0, API v2.0, Component v1.6.4)
MCA plm: rsh (MCA v2.0, API v2.0, Component v1.6.4)
MCA plm: slurm (MCA v2.0, API v2.0, Component v1.6.4)
MCA filem: rsh (MCA v2.0, API v2.0, Component v1.6.4)
MCA errmgr: default (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: env (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: hnp (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: singleton (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: slave (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: slurm (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: slurmd (MCA v2.0, API v2.0, Component v1.6.4)
MCA ess: tool (MCA v2.0, API v2.0, Component v1.6.4)
MCA grpcomm: bad (MCA v2.0, API v2.0, Component v1.6.4)
MCA grpcomm: basic (MCA v2.0, API v2.0, Component v1.6.4)
MCA grpcomm: hier (MCA v2.0, API v2.0, Component v1.6.4)
MCA notifier: command (MCA v2.0, API v1.0, Component v1.6.4)
MCA notifier: syslog (MCA v2.0, API v1.0, Component v1.6.4)
(2013/05/16 9:12), Jeff Squyres (jsquyres) wrote:
I am unable to replicate your error -- 1.6.4 has MPI_REDUCE defined on MPI_SUM
properly.
Are you absolutely sure you're using OMPI 1.6.4 on all nodes?
Try this:
mpirun ... ompi_info
(insert whatever hostfile and -np value you're using for your fortran test) and
see what is reported as the OMPI version from other nodes.
On May 15, 2013, at 7:46 AM, Hayato KUNIIE <kuni...@oita.email.ne.jp> wrote:
I using Ver, 1.6.4 in all nodes.
(2013/05/15 7:10), Jeff Squyres (jsquyres) wrote:
Are you sure that you have exactly the same version of Open MPI on all your
nodes?
On May 14, 2013, at 11:39 AM, Hayato KUNIIE <kuni...@oita.email.ne.jp> wrote:
Hello I'm kuni255
I build bewulf type PC Cluster (Cent OS release 6.4). And I studing
about MPI.(Open MPI Ver.1.6.4) I tried following sample which using
MPI_REDUCE.
Then, Error occured.
This cluster system consist of one head node and 2 slave nodes.
And sharing home directory in head node by NFS. so Open MPI is installed
each nodes.
When I test this program on only head node, program is run correctly.
and output result.
But When I test this program on only slave node, same error occured.
Please tell me, good idea : )
Error message
[bwslv01:30793] *** An error occurred in MPI_Reduce: the reduction
operation MPI_SUM is not defined on the MPI_INTEGER datatype
[bwslv01:30793] *** on communicator MPI_COMM_WORLD
[bwslv01:30793] *** MPI_ERR_OP: invalid reduce operation
[bwslv01:30793] *** MPI_ERRORS_ARE_FATAL: your MPI job will now abort
--------------------------------------------------------------------------
mpirun has exited due to process rank 1 with PID 30793 on
node bwslv01 exiting improperly. There are two reasons this could occur:
1. this process did not call "init" before exiting, but others in
the job did. This can cause a job to hang indefinitely while it waits
for all processes to call "init". By rule, if one process calls "init",
then ALL processes must call "init" prior to termination.
2. this process called "init", but exited without calling "finalize".
By rule, all processes that call "init" MUST call "finalize" prior to
exiting or it will be considered an "abnormal termination"
This may have caused other processes in the application to be
terminated by signals sent by mpirun (as reported here).
--------------------------------------------------------------------------
[bwhead.clnet:02147] 1 more process has sent help message
help-mpi-errors.txt / mpi_errors_are_fatal
[bwhead.clnet:02147] Set MCA parameter "orte_base_help_aggregate" to 0
to see all help / error messages
Fortran90 source code
include 'mpif.h'
parameter(nmax=12)
integer n(nmax)
call mpi_init(ierr)
call mpi_comm_size(MPI_COMM_WORLD, isize, ierr)
call mpi_comm_rank(MPI_COMM_WORLD, irank, ierr)
ista=irank*(nmax/isize) + 1
iend=ista+(nmax/isize-1)
isum=0
do i=1,nmax
n(i) = i
isum = isum + n(i)
end do
call mpi_reduce(isum, itmp, 1, MPI_INTEGER, MPI_SUM,
& 0, MPI_COMM_WORLD, ierr)
if (irank == 0) then
isum=itmp
WRITE(*,*) isum
endif
call mpi_finalize(ierr)
end
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
