Re: [OMPI users] Big jump from OFED 1.5.4.1 -> recent (stable). Any suggestions?

tack is ³not based on any direct OFED version²? Doesn¹t Redhat just ship an old OFED build, or they do their own changes to it like to the kernel? -- Grigory Shamov Westgrid/ComputeCanada Site Lead University of Manitoba E2-588 EITC Building, (204) 474-9625

Re: [OMPI users] oob-tcp error (warning?) message

: hwloc_set_cpubind returned "Error" for bitmap "0" Location: ../../../../../openmpi-1.10.0/orte/mca/odls/default/odls_default_module.c:5 51 -- Grigory Shamov HPC Analist, Westgrid/ComputeCanada Site Lead University of Manitoba E2-588 EITC Building, (204) 474-9625

[OMPI users] oob-tcp error (warning?) message

ng job; we use OpenMPI 1.6.5 and OFED 2.4 on CentOS 6. -- Grigory Shamov HPC Analist, Westgrid/ComputeCanada Site Lead University of Manitoba E2-588 EITC Building, (204) 474-9625

Re: [OMPI users] Process binding with SLURM and 'heterogeneous' nodes

efault_module.c:5 51 -- Grigory Shamov Westgrid/ComputeCanada Site Lead University of Manitoba E2-588 EITC Building, (204) 474-9625 On 15-10-02 10:25 AM, "users on behalf of Marcin Krotkiewski" <users-boun...@open-mpi.org on behalf of marcin.krotkiew...@gmail.com> wrote:

Re: [OMPI users] Setting bind-to none as default via environment?

Thanks, I guess it be hwloc-base-binding-policy = in the file. Found it. -- Grigory Shamov Westgrid/ComputeCanada Site Lead University of Manitoba E2-588 EITC Building, (204) 474-9625 From: users <users-boun...@open-mpi.org<mailto:users-boun...@open-mpi.org>> on behalf of Nick Pap

Re: [OMPI users] Using OpenMPI (1.8, 1.10) with Mellanox MXM, ulimits ?

Hi Thomas, Thank you for the suggestion! Will try it. -- Grigory Shamov On 15-09-30 6:57 AM, "users on behalf of Thomas Jahns" <users-boun...@open-mpi.org on behalf of ja...@dkrz.de> wrote: >Hello, > >On 09/28/15 18:36, Grigory Shamov wrote: >> The question

Re: [OMPI users] Using OpenMPI (1.8, 1.10) with Mellanox MXM, ulimits ?

rrays are allocated on the stack (DEFAULT)". Which leads potentially to the code using large stack, way over 10Mb. -- Grigory Shamov Westgrid/ComputeCanada Site Lead University of Manitoba E2-588 EITC Building, (204) 474-9625 On 15-09-30 5:19 AM, "users on behalf of Dave Love"

Re: [OMPI users] Using OpenMPI (1.8, 1.10) with Mellanox MXM, ulimits ?

of MPI's and compilers is already quite a task; making one MPI build for this app and another MPI build for that app is sort of, not really practical, nor is educating users to set ulimits and mca parameters per each job. -- Grigory Shamov Westgrid/ComputeCanada Site Lead University of Manitoba E2

Re: [OMPI users] Using OpenMPI (1.8, 1.10) with Mellanox MXM, ulimits ?

start (we set the ulimits in Torque). Is it known (I know every application is different ) how much costs, performance-wise, to have MXM with good ulimits vs unlimited ulimits, vs not using MXM at all? -- Grigory Shamov Westgrid/ComputeCanada Site Lead University of Manitoba E2-588 EITC Building

[OMPI users] Using OpenMPI (1.8, 1.10) with Mellanox MXM, ulimits ?

an experience running recent OpenMPI with MXM enabled, and what kind of ulimits do you have? Any suggestions/comments appreciated, thanks! -- Grigory Shamov Westgrid/ComputeCanada Site Lead University of Manitoba E2-588 EITC Building, (204) 474-9625