Re: [OMPI users] change in behaviour 1.6 -> 1.8 under sge

2014-11-05 Thread Dave Love
Ralph Castain writes: > I confirmed that things are working as intended. I could have been more explicit saying so before. > If you have 12 cores on a machine, and you do > > mpirun -map-by socket:PE=2 > > we will execute 6 copies of foo on the node because 12 cores/2pe/core = 6 > procs. For

Re: [OMPI users] change in behaviour 1.6 -> 1.8 under sge

2014-11-04 Thread Ralph Castain
I confirmed that things are working as intended. If you have 12 cores on a machine, and you do mpirun -map-by socket:PE=2 we will execute 6 copies of foo on the node because 12 cores/2pe/core = 6 procs. As I said, we believe the prior series were doing this incorrectly, and the patch used on

Re: [OMPI users] change in behaviour 1.6 -> 1.8 under sge

2014-11-04 Thread Dave Love
Ralph Castain writes: > If you only have one allocated PE on a node, then mpirun will > correctly tell you that it can’t launch with PE>1 as there aren’t > enough resources to meet your request. IIRC, we may have been ignoring > this under SGE and running as many procs as we wanted on an allocate

Re: [OMPI users] change in behaviour 1.6 -> 1.8 under sge

2014-11-04 Thread Dave Love
I wrote: > #$ -l exclusive > export OMP_NUM_THREADS=2 > exec mpirun --loadbalance --cpus-per-proc $OMP_NUM_THREADS --np > $(($NSLOTS/$OMP_NUM_THREADS)) ... I should have said core binding is the default here [so Intel MPI does

Re: [OMPI users] change in behaviour 1.6 -> 1.8 under sge

2014-11-03 Thread Dave Love
Mark Dixon writes: > Hi there, > > We've started looking at moving to the openmpi 1.8 branch from 1.6 on > our CentOS6/Son of Grid Engine cluster and noticed an unexpected > difference when binding multiple cores to each rank. > > Has openmpi's definition 'slot' changed between 1.6 and 1.8? You

Re: [OMPI users] change in behaviour 1.6 -> 1.8 under sge

2014-11-03 Thread Ralph Castain
> On Nov 3, 2014, at 4:54 AM, Mark Dixon wrote: > > Hi there, > > We've started looking at moving to the openmpi 1.8 branch from 1.6 on our > CentOS6/Son of Grid Engine cluster and noticed an unexpected difference when > binding multiple cores to each rank. > > Has openmpi's definition 'slot

[OMPI users] change in behaviour 1.6 -> 1.8 under sge

2014-11-03 Thread Mark Dixon
Hi there, We've started looking at moving to the openmpi 1.8 branch from 1.6 on our CentOS6/Son of Grid Engine cluster and noticed an unexpected difference when binding multiple cores to each rank. Has openmpi's definition 'slot' changed between 1.6 and 1.8? It used to mean ranks, but now it