Dear all,
I have noticed that SLURM schedules several jobs from a job array on the
same node with the same start time and end time.
Each of these jobs requires the full node. You can see the squeue output
below:
JOBID PARTITION ST START_TIME NODES
SCHEDNODES NODELIST(REASON)
124841_1 cpu PD 2021-12-11T03:58:00 1
cn06 (Priority)
124841_2 cpu PD 2021-12-11T03:58:00 1
cn06 (Priority)
124841_3 cpu PD 2021-12-11T03:58:00 1
cn06 (Priority)
124841_4 cpu PD 2021-12-11T03:58:00 1
cn06 (Priority)
124841_5 cpu PD 2021-12-11T03:58:00 1
cn06 (Priority)
124841_6 cpu PD 2021-12-11T03:58:00 1
cn06 (Priority)
124841_7 cpu PD 2021-12-11T03:58:00 1
cn06 (Priority)
124841_8 cpu PD 2021-12-11T03:58:00 1
cn06 (Priority)
124841_9 cpu PD 2021-12-11T03:58:00 1
cn06 (Priority)
Is this a bug or am I missing something? Is this because the jobs have
the same JOBID and are still in pending state? I am aware that the jobs
will not actually all run on the same node at the same time and that the
scheduler somehow takes into account that this job array has 9 jobs that
will need 9 nodes. I am creating a timeline with the start time of all
jobs and when the job array jobs will start running no other jobs are
set to run on the remaining nodes (so it "saves" the other nodes for the
jobs of the array even if they are all scheduled to run on the same node
based on squeue or scontrol).
Regards,
Thekla Loizou
HPC Systems Engineer
The Cyprus Institute
