Re: [OMPI users] scheduling to real cores, not using hyperthreading (openmpi-2.1.0)

I’m afraid none of the current options is going to do that right now. I’ll put a note on my to-do list to look at this, but I can’t promise when I’ll get to it. > On Apr 24, 2017, at 3:13 AM, Heinz-Ado Arnolds > wrote: > > Dear Ralph, > > thanks for this new

Re: [OMPI users] scheduling to real cores, not using hyperthreading (openmpi-2.1.0)

Dear Ralph, thanks for this new hint. Unfortunately I don't see how that would fulfill all my requirements: I like to have 8 OpenMPI jobs for 2 nodes -> 4 OpenMPI jobs per node -> 2 per socket, each executing one OpenMP job with 5 threads mpirun -np 8 --map-by ppr:4:node:pe=5 ... How can

Re: [OMPI users] scheduling to real cores, not using hyperthreading (openmpi-2.1.0)

Sorry for delayed response. I’m glad that option solved the problem. We’ll have to look at that configure option - shouldn’t be too hard. As for the mapping you requested - no problem! Here’s the cmd line: mpirun --map-by ppr:1:core --bind-to hwthread Ralph > On Apr 19, 2017, at 2:51 AM,

Re: [OMPI users] scheduling to real cores, not using hyperthreading (openmpi-2.1.0)

You can always specify a particular number of cpus to use for each process by adding it to the map-by directive: mpirun -np 8 --map-by ppr:2:socket:pe=5 --use-hwthread-cpus -report-bindings --mca plm_rsh_agent "qrsh" ./myid would map 2 processes to each socket, binding each process to 5 HTs

Re: [OMPI users] scheduling to real cores, not using hyperthreading (openmpi-2.1.0)

Dear Gilles, thanks a lot for your response! 1. You're right, my stupid error, I forgot the "export" of OMP_PROC_BIND in my job script. Now this example is working nearly as expected: [pascal-1-07:25617] MCW rank 0 bound to socket 0[core 0[hwt 0-1]], socket 0[core 1[hwt 0-1]], socket

Re: [OMPI users] scheduling to real cores, not using hyperthreading (openmpi-2.1.0)

Heinz-Ado, it seems the OpenMP runtime did *not* bind the OMP threads at all as requested, and the root cause could be the OMP_PROC_BIND environment variable was not propagated can you try mpirun -x OMP_PROC_BIND ... and see if it helps ? Cheers, On 4/13/2017 12:23 AM, Heinz-Ado

Re: [OMPI users] scheduling to real cores, not using hyperthreading (openmpi-2.1.0)

Dear Gilles, thanks for your answer. - compiler: gcc-6.3.0 - OpenMP environment vars: OMP_PROC_BIND=true, GOMP_CPU_AFFINITY not set - hyperthread a given OpenMP thread is on: it's printed in the output below as a 3-digit number after the first ",", read by sched_getcpu() in the OpenMP test

Re: [OMPI users] scheduling to real cores, not using hyperthreading (openmpi-2.1.0)

That should be a two steps tango - Open MPI bind a MPI task to a socket - the OpenMP runtime bind OpenMP threads to cores (or hyper threads) inside the socket assigned by Open MPI which compiler are you using ? do you set some environment variables to direct OpenMP to bind threads ? Also, how do

Re: [OMPI users] scheduling to real cores, not using hyperthreading (openmpi-2.1.0)

Open MPI isn’t doing anything wrong - it is doing exactly what it should, and exactly what you are asking it to do. The problem you are having is that OpenMP isn’t placing the threads exactly where you would like inside the process-level “envelope” that Open MPI has bound the entire process to.

Re: [OMPI users] scheduling to real cores, not using hyperthreading (openmpi-2.1.0)

Dear rhc, to make it more clear what I try to achieve, I collected some examples for several combinations of command line options. Would be great if you find time to look to these below. The most promise one is example "4". I'd like to have 4 MPI jobs starting 1 OpenMP job each with 10

Re: [OMPI users] scheduling to real cores, not using hyperthreading (openmpi-2.1.0)

I’m not entirely sure I understand your reference to “real cores”. When we bind you to a core, we bind you to all the HT’s that comprise that core. So, yes, with HT enabled, the binding report will list things by HT, but you’ll always be bound to the full core if you tell us bind-to core The

[OMPI users] scheduling to real cores, not using hyperthreading (openmpi-2.1.0)

Dear OpenMPI users & developers, I'm trying to distribute my jobs (with SGE) to a machine with a certain number of nodes, each node having 2 sockets, each socket having 10 cores & 10 hyperthreads. I like to use only the real cores, no hyperthreading. lscpu -a -e CPU NODE SOCKET CORE