That's a pretty old OMPI version, and we don't really support it any longer. However, I can provide some advice:
* have you tried running the simple "hello_c" example we provide? This would at least tell you if the problem is in your app, which is what I'd expect given your description * try using gdb (or pick your debugger) to look at the corefile and see where it is failing I'd also suggest updating OMPI to the 1.6.5 or 1.8.1 versions, but I doubt that's the issue behind this problem. On Jul 19, 2014, at 1:05 AM, Lane, William <william.l...@cshs.org> wrote: > I'm getting consistent errors of the form: > > "mpirun noticed that process rank 3 with PID 802 on node csclprd3-0-8 exited > on signal 11 (Segmentation fault)." > > whenever I request more than 28 slots. These > errors even occur when I run mpirun locally > on a compute node that has 32 slots (8 cores, 16 with hyperthreading). > > When I run less than 28 slots I have no problems whatsoever. > > OS: > CentOS release 6.3 (Final) > > openMPI information: > Package: Open MPI mockbu...@c6b8.bsys.dev.centos.org > Distribution > Open MPI: 1.5.4 > Open MPI SVN revision: r25060 > Open MPI release date: Aug 18, 2011 > Open RTE: 1.5.4 > Open RTE SVN revision: r25060 > Open RTE release date: Aug 18, 2011 > OPAL: 1.5.4 > OPAL SVN revision: r25060 > OPAL release date: Aug 18, 2011 > Ident string: 1.5.4 > Prefix: /usr/lib64/openmpi > Configured architecture: x86_64-unknown-linux-gnu > Configure host: c6b8.bsys.dev.centos.org > Configured by: mockbuild > Configured on: Fri Jun 22 06:42:03 UTC 2012 > Configure host: c6b8.bsys.dev.centos.org > Built by: mockbuild > Built on: Fri Jun 22 06:46:48 UTC 2012 > Built host: c6b8.bsys.dev.centos.org > C bindings: yes > C++ bindings: yes > Fortran77 bindings: yes (all) > Fortran90 bindings: yes > Fortran90 bindings size: small > C compiler: gcc > C compiler absolute: /usr/bin/gcc > C compiler family name: GNU > C compiler version: 4.4.6 > C++ compiler: g++ > C++ compiler absolute: /usr/bin/g++ > Fortran77 compiler: gfortran > Fortran77 compiler abs: /usr/bin/gfortran > Fortran90 compiler: gfortran > Fortran90 compiler abs: /usr/bin/gfortran > C profiling: yes > C++ profiling: yes > Fortran77 profiling: yes > Fortran90 profiling: yes > C++ exceptions: no > Thread support: posix (MPI_THREAD_MULTIPLE: no, progress: no) > Sparse Groups: no > Internal debug support: no > MPI interface warnings: no > MPI parameter check: runtime > Memory profiling support: no > Memory debugging support: no > libltdl support: yes > Heterogeneous support: no > mpirun default --prefix: no > MPI I/O support: yes > MPI_WTIME support: gettimeofday > Symbol vis. support: yes > MPI extensions: affinity example > FT Checkpoint support: no (checkpoint thread: no) > MPI_MAX_PROCESSOR_NAME: 256 > MPI_MAX_ERROR_STRING: 256 > MPI_MAX_OBJECT_NAME: 64 > MPI_MAX_INFO_KEY: 36 > MPI_MAX_INFO_VAL: 256 > MPI_MAX_PORT_NAME: 1024 > MPI_MAX_DATAREP_STRING: 128 > MCA backtrace: execinfo (MCA v2.0, API v2.0, Component v1.5.4) > MCA memchecker: valgrind (MCA v2.0, API v2.0, Component v1.5.4) > MCA memory: linux (MCA v2.0, API v2.0, Component v1.5.4) > MCA paffinity: hwloc (MCA v2.0, API v2.0, Component v1.5.4) > MCA carto: auto_detect (MCA v2.0, API v2.0, Component v1.5.4) > MCA carto: file (MCA v2.0, API v2.0, Component v1.5.4) > MCA maffinity: first_use (MCA v2.0, API v2.0, Component v1.5.4) > MCA maffinity: libnuma (MCA v2.0, API v2.0, Component v1.5.4) > MCA timer: linux (MCA v2.0, API v2.0, Component v1.5.4) > MCA installdirs: env (MCA v2.0, API v2.0, Component v1.5.4) > MCA installdirs: config (MCA v2.0, API v2.0, Component v1.5.4) > MCA dpm: orte (MCA v2.0, API v2.0, Component v1.5.4) > MCA pubsub: orte (MCA v2.0, API v2.0, Component v1.5.4) > MCA allocator: basic (MCA v2.0, API v2.0, Component v1.5.4) > MCA allocator: bucket (MCA v2.0, API v2.0, Component v1.5.4) > MCA coll: basic (MCA v2.0, API v2.0, Component v1.5.4) > MCA coll: hierarch (MCA v2.0, API v2.0, Component v1.5.4) > MCA coll: inter (MCA v2.0, API v2.0, Component v1.5.4) > MCA coll: self (MCA v2.0, API v2.0, Component v1.5.4) > MCA coll: sm (MCA v2.0, API v2.0, Component v1.5.4) > MCA coll: sync (MCA v2.0, API v2.0, Component v1.5.4) > MCA coll: tuned (MCA v2.0, API v2.0, Component v1.5.4) > MCA mpool: fake (MCA v2.0, API v2.0, Component v1.5.4) > MCA mpool: rdma (MCA v2.0, API v2.0, Component v1.5.4) > MCA mpool: sm (MCA v2.0, API v2.0, Component v1.5.4) > MCA pml: bfo (MCA v2.0, API v2.0, Component v1.5.4) > MCA pml: csum (MCA v2.0, API v2.0, Component v1.5.4) > MCA pml: ob1 (MCA v2.0, API v2.0, Component v1.5.4) > MCA pml: v (MCA v2.0, API v2.0, Component v1.5.4) > MCA bml: r2 (MCA v2.0, API v2.0, Component v1.5.4) > MCA rcache: vma (MCA v2.0, API v2.0, Component v1.5.4) > MCA btl: ofud (MCA v2.0, API v2.0, Component v1.5.4) > MCA btl: openib (MCA v2.0, API v2.0, Component v1.5.4) > MCA btl: self (MCA v2.0, API v2.0, Component v1.5.4) > MCA btl: sm (MCA v2.0, API v2.0, Component v1.5.4) > MCA btl: tcp (MCA v2.0, API v2.0, Component v1.5.4) > MCA topo: unity (MCA v2.0, API v2.0, Component v1.5.4) > MCA osc: pt2pt (MCA v2.0, API v2.0, Component v1.5.4) > MCA osc: rdma (MCA v2.0, API v2.0, Component v1.5.4) > MCA iof: hnp (MCA v2.0, API v2.0, Component v1.5.4) > MCA iof: orted (MCA v2.0, API v2.0, Component v1.5.4) > MCA iof: tool (MCA v2.0, API v2.0, Component v1.5.4) > MCA oob: tcp (MCA v2.0, API v2.0, Component v1.5.4) > MCA odls: default (MCA v2.0, API v2.0, Component v1.5.4) > MCA ras: cm (MCA v2.0, API v2.0, Component v1.5.4) > MCA ras: gridengine (MCA v2.0, API v2.0, Component v1.5.4) > MCA ras: loadleveler (MCA v2.0, API v2.0, Component v1.5.4) > MCA ras: slurm (MCA v2.0, API v2.0, Component v1.5.4) > MCA rmaps: load_balance (MCA v2.0, API v2.0, Component v1.5.4) > MCA rmaps: rank_file (MCA v2.0, API v2.0, Component v1.5.4) > MCA rmaps: resilient (MCA v2.0, API v2.0, Component v1.5.4) > MCA rmaps: round_robin (MCA v2.0, API v2.0, Component v1.5.4) > MCA rmaps: seq (MCA v2.0, API v2.0, Component v1.5.4) > MCA rmaps: topo (MCA v2.0, API v2.0, Component v1.5.4) > MCA rml: oob (MCA v2.0, API v2.0, Component v1.5.4) > MCA routed: binomial (MCA v2.0, API v2.0, Component v1.5.4) > MCA routed: cm (MCA v2.0, API v2.0, Component v1.5.4) > MCA routed: direct (MCA v2.0, API v2.0, Component v1.5.4) > MCA routed: linear (MCA v2.0, API v2.0, Component v1.5.4) > MCA routed: radix (MCA v2.0, API v2.0, Component v1.5.4) > MCA routed: slave (MCA v2.0, API v2.0, Component v1.5.4) > MCA plm: rsh (MCA v2.0, API v2.0, Component v1.5.4) > MCA plm: rshd (MCA v2.0, API v2.0, Component v1.5.4) > MCA plm: slurm (MCA v2.0, API v2.0, Component v1.5.4) > MCA filem: rsh (MCA v2.0, API v2.0, Component v1.5.4) > MCA errmgr: default (MCA v2.0, API v2.0, Component v1.5.4) > MCA ess: env (MCA v2.0, API v2.0, Component v1.5.4) > MCA ess: hnp (MCA v2.0, API v2.0, Component v1.5.4) > MCA ess: singleton (MCA v2.0, API v2.0, Component v1.5.4) > MCA ess: slave (MCA v2.0, API v2.0, Component v1.5.4) > MCA ess: slurm (MCA v2.0, API v2.0, Component v1.5.4) > MCA ess: slurmd (MCA v2.0, API v2.0, Component v1.5.4) > MCA ess: tool (MCA v2.0, API v2.0, Component v1.5.4) > MCA grpcomm: bad (MCA v2.0, API v2.0, Component v1.5.4) > MCA grpcomm: basic (MCA v2.0, API v2.0, Component v1.5.4) > MCA grpcomm: hier (MCA v2.0, API v2.0, Component v1.5.4) > MCA notifier: command (MCA v2.0, API v1.0, Component v1.5.4) > MCA notifier: smtp (MCA v2.0, API v1.0, Component v1.5.4) > MCA notifier: syslog (MCA v2.0, API v1.0, Component v1.5.4) > > IMPORTANT WARNING: This message is intended for the use of the person or > entity to which it is addressed and may contain information that is > privileged and confidential, the disclosure of which is governed by > applicable law. If the reader of this message is not the intended recipient, > or the employee or agent responsible for delivering it to the intended > recipient, you are hereby notified that any dissemination, distribution or > copying of this information is STRICTLY PROHIBITED. If you have received this > message in error, please notify us immediately by calling (310) 423-6428 and > destroy the related message. Thank You for your cooperation. > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/07/24815.php
