Re: [OMPI users] New ib locked pages behavior?

2014-10-24 Thread Jeff Squyres (jsquyres)
On Oct 22, 2014, at 3:37 AM, r...@q-leap.de wrote: > I've commented in detail on this (non-)issue on 2014-08-20: > > http://www.open-mpi.org/community/lists/users/2014/08/25090.php > > A change in the FAQ and a fix in the code would really be nice > at this stage. Thanks for the reminder; I've

Re: [OMPI users] New ib locked pages behavior?

2014-10-23 Thread rf
> "Bill" == Bill Broadley writes: Hi Bill, >> I've commented in detail on this (non-)issue on 2014-08-20: >> http://www.open-mpi.org/community/lists/users/2014/08/25090.php Bill> I read that. It seems pretty clear what the problem is, but Bill> not so clear on what a user

Re: [OMPI users] New ib locked pages behavior?

2014-10-22 Thread Bill Broadley
On 10/22/2014 12:37 AM, r...@q-leap.de wrote: >> "Bill" == Bill Broadley writes: > > It seems the half-life period of knowledge on the list has decayed to > two weeks on the list :) > > I've commented in detail on this (non-)issue on 2014-08-20: > > http://www.open-mpi.org/community/lists/u

Re: [OMPI users] New ib locked pages behavior?

2014-10-22 Thread rf
> "Bill" == Bill Broadley writes: It seems the half-life period of knowledge on the list has decayed to two weeks on the list :) I've commented in detail on this (non-)issue on 2014-08-20: http://www.open-mpi.org/community/lists/users/2014/08/25090.php A change in the FAQ and a fix in the

Re: [OMPI users] New ib locked pages behavior?

2014-10-22 Thread Bill Broadley
On 10/21/2014 05:38 PM, Gus Correa wrote: > Hi Bill > > I have 2.6.X CentOS stock kernel. Heh, wow, quite a blast from the past. > I set both parameters. > It works. Yes, for kernels that old I had it working fine. > Maybe the parameter names may changed in 3.X kernels? > (Which is really bad

Re: [OMPI users] New ib locked pages behavior?

2014-10-21 Thread Gus Correa
Hi Bill I have 2.6.X CentOS stock kernel. I set both parameters. It works. Maybe the parameter names may changed in 3.X kernels? (Which is really bad ...) You could check if there is more information in: /sys/module/mlx4_core/parameters/ There seems to be a thread on the list about this (but ap

Re: [OMPI users] New ib locked pages behavior?

2014-10-21 Thread Bill Broadley
On 10/21/2014 04:18 PM, Gus Correa wrote: > Hi Bill > > Maybe you're missing these settings in /etc/modprobe.d/mlx4_core.conf ? > > http://www.open-mpi.org/faq/?category=openfabrics#ib-low-reg-mem Ah, that helped. Although: /lib/modules/3.13.0-36-generic/kernel/drivers/net/ethernet/mellanox/mlx

Re: [OMPI users] New ib locked pages behavior?

2014-10-21 Thread Gus Correa
Hi Bill Maybe you're missing these settings in /etc/modprobe.d/mlx4_core.conf ? http://www.open-mpi.org/faq/?category=openfabrics#ib-low-reg-mem I hope this helps, Gus Correa On 10/21/2014 06:36 PM, Bill Broadley wrote: I've setup several clusters over the years with OpenMPI. I often get th

[OMPI users] New ib locked pages behavior?

2014-10-21 Thread Bill Broadley
I've setup several clusters over the years with OpenMPI. I often get the below error: WARNING: It appears that your OpenFabrics subsystem is configured to only allow registering part of your physical memory. This can cause MPI jobs to run with erratic performance, hang, and/or crash.