Hi, Just for later reference, it turns out I have only to add thing like this in the submission script: ###SBATCH --ntasks-per-socket=4 export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK} export OMP_PROC_BIND="spread" ## make cores in order inside a task srun --cpu-bind=v,cores -m block:block ./code
Thanks, Chris On Fri, Apr 19, 2019 at 9:22 AM Lincy <lincy...@gmail.com> wrote: > Hi all, > I have a cpu allocation problem on the 2 socket 18-core machine, > with CPUSpecList=17,35 reserved. > Ideally, if I submit a job with two 17-core tasks in a node, it allocates > each task on a single NUMA node. > > # *srun -N1 -n2 --cpus-per-task=**17** numactl -s *physcpubind: *0 1 2 > 3 4 5 6 7 8 9 10 11 12 13 14 15 16* cpubind: 0 membind: 0 1 physcpubind: *18 > 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34* cpubind: 1 membind: 0 1 > However, a job with two 16-core tasks show a bad allocation (cover two > numa nodes) even if I enforce it by --cpu-bind=mask_cpu:0xffff,0x3fffc0000 > > # *srun -N1 -n2 --cpus-per-task=**16** numactl -s*, physcpubind: *0 1 2 > 3 4 5 6 7 8 9 10 11 12 13 14 15* cpubind: 0 membind: 0 1 physcpubind: *16 > 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32* cpubind: 0 1 membind: 0 1 > How should I allocate it on 0-15, 18-33 for example? > > Many thanks, > Chris >