[OMPI users] openmpi single node jobs using btl openib

2017-02-06 Thread Jingchao Zhang
Hi, We recently noticed openmpi is using btl openib over self,sm for single node jobs, which has caused performance degradation for some applications, e.g. 'cp2k'. For opempi version 2.0.1, our test shows single node 'cp2k' job using openib is ~25% slower than using self,sm. We advise users do

Re: [OMPI users] openmpi single node jobs using btl openib

2017-02-07 Thread Tobias Kloeffel
Hello Jingchao, try to use -mca mpi_leave_pinned 0, also for multinode jobs. kind regards, Tobias Klöffel On 02/06/2017 09:38 PM, Jingchao Zhang wrote: Hi, We recently noticed openmpi is using btl openib over self,sm for single node jobs, which has caused performance degradation for some a

Re: [OMPI users] openmpi single node jobs using btl openib

2017-02-07 Thread Jingchao Zhang
jobs? Thanks, Jingchao From: users on behalf of Tobias Kloeffel Sent: Tuesday, February 7, 2017 2:54:46 AM To: Open MPI Users Subject: Re: [OMPI users] openmpi single node jobs using btl openib Hello Jingchao, try to use -mca mpi_leave_pinned 0, also for multinode jo

Re: [OMPI users] openmpi single node jobs using btl openib

2017-02-07 Thread Jeff Squyres (jsquyres)
Jingchao > From: users on behalf of Tobias Kloeffel > > Sent: Tuesday, February 7, 2017 2:54:46 AM > To: Open MPI Users > Subject: Re: [OMPI users] openmpi single node jobs using btl openib > > Hello Jingchao, > try to use -mca mpi_leave_pinned 0, also for multinode jobs. &

Re: [OMPI users] openmpi single node jobs using btl openib

2017-02-07 Thread Jingchao Zhang
, Jingchao From: users on behalf of Jeff Squyres (jsquyres) Sent: Tuesday, February 7, 2017 2:14:40 PM To: Open MPI User's List Subject: Re: [OMPI users] openmpi single node jobs using btl openib Can you try upgrading to Open MPI v2.0.2? We just released that

Re: [OMPI users] openmpi single node jobs using btl openib

2017-02-07 Thread Gilles Gouaillardet
:* Tuesday, February 7, 2017 2:14:40 PM *To:* Open MPI User's List *Subject:* Re: [OMPI users] openmpi single node jobs using btl openib Can you try upgrading to Open MPI v2.0.2? We just released that last week with a bunch of bug fixes. > On Feb 7, 2017, at 3:07 PM, Jingchao Zhan

Re: [OMPI users] openmpi single node jobs using btl openib

2017-02-07 Thread Nathan Hjelm
with-cma > > > > So the question remains why OpenMPI choose openib over self,sm for single > > node jobs? Isn't there a mechanism to differentiate btl networks for > > single/multi-node jobs? > > > > Thanks, > > Jingchao > > From: users on

Re: [OMPI users] openmpi single node jobs using btl openib

2017-02-08 Thread Cabral, Matias A
Users Subject: Re: [OMPI users] openmpi single node jobs using btl openib Hi Gilles, - btl/vader has a higher exclusivity than btl/sm, so bottom line, vader should be used instead of sm Yes, I believe vader is being used in our case. I should say self,vader in my previous posts. - is your