Re: [OMPI users] large memory usage and hangs when preconnecting beyond 1000 cpus

2014-10-30 Thread Marshall Ward
elm > HPC-5, LANL > > On Tue, Oct 21, 2014 at 01:12:21PM +1100, Marshall Ward wrote: >> Thanks, it's at least good to know that the behaviour isn't normal! >> >> Could it be some sort of memory leak in the call? The code in >> >> ompi/runtime/ompi

Re: [OMPI users] large memory usage and hangs when preconnecting beyond 1000 cpus

2014-10-20 Thread Marshall Ward
messages? I'm trying to see if valgrind can see anything, but nothing from ompi_init_preconnect_mpi is coming up (although there are some other warnings). On Sun, Oct 19, 2014 at 2:37 AM, Ralph Castain <r...@open-mpi.org> wrote: > >> On Oct 17, 2014, at 3:37 AM, Marshall W

[OMPI users] large memory usage and hangs when preconnecting beyond 1000 cpus

2014-10-17 Thread Marshall Ward
I currently have a numerical model that, for reasons unknown, requires preconnection to avoid hanging on an initial MPI_Allreduce call. But when we try to scale out beyond around 1000 cores, we are unable to get past MPI_Init's preconnection phase. To test this, I have a basic C program