Hi Anne,
On 28/3/19 1:33 pm, Anne M. Hammond wrote:
So one has to be explicit about ntasks-per-core on the command line
or in the sbatch file, even though ThreadsPerCore is in slurm.conf.
If I remember correctly by default Slurm will only launch one task per
physical core, hence why you need
You probably shouldn't use "CPUs=24" and "Sockets=2 CoresPerSocket=6
ThreadsPerCore=2" at the same time.
CPUs=24 (or more exactly Tasks=24, though this setting does not exist) follows
directly from the other settings.
From slurm.conf man page:
CPUs
Number of logical processors on the
Thanks Jeffrey and Chris.
This does not work:
srun -N 1 -n 24 --pty bash -i
But this works:
srun -N 1 -n 24 --ntasks-per-core=2 --pty bash -i
So one has to be explicit about ntasks-per-core on the command line or in the
sbatch file, even though ThreadsPerCore is in slurm.conf.
> On Mar
On 27/3/19 1:00 pm, Anne M. Hammond wrote:
NodeName=fl[01-04] CPUs=24 RealMemory=4 Sockets=2 CoresPerSocket=6
ThreadsPerCore=2 State=UNKNOWN
This will give you 12 tasks per node, each task with 2 thread units.
--
Chris Samuel : http://www.csamuel.org/ : Berkeley, CA, USA
Thanks. A second user cannot allocate any tasks on the node
which is running 12 processes.
So it does look like it slurm is tieing processes to physical cores.
Further interesting is that top shows all 24 Cpus at ~95%:
%Cpu0 : 93.4 us, 0.7 sy, 0.0 ni, 6.0 id, 0.0 wa, 0.0 hi, 0.0 si,
Can a second user allocate anything on node fl01 after the first user requests
their 12 tasks per node? If not, then it looks like tasks are being tied to
physical cores, and not a hyperthreaded version of a core.
--
Mike Renfro, PhD / HPC Systems Administrator, Information Technology Services
We are just getting started with slurm here. We have
slurm 18.08.6-2
/etc/slurm/slurm.conf:
NodeName=fl[01-04] CPUs=24 RealMemory=4 Sockets=2 CoresPerSocket=6
ThreadsPerCore=2 State=UNKNOWN
Cannot allocate ntasks-per-node:
[hammond@hydrogen VSim-9.0]$ srun -N 1 --ntasks-per-node=24
