At least from my experience wonky things can happen with slurm
(especially if you have thread affinity on) if you don't rightly divide
between -n and -c. In general I've been telling our users that -c is
for threaded applications and -n is for rank based parallelism. This
way the thread affinity works out properly.
-Paul Edmon-
On 02/22/2018 08:50 AM, Christopher Benjamin Coffey wrote:
Loris,
It’s simple, tell folks only to use -n for mpi jobs, and -c otherwise (default).
It’s a big deal if folks use -n when it’s not an mpi program. This is because
the non mpi program is launched n times (instead of once with internal threads)
and will stomp over logs and output files (uncoordinated) leading to poor
performance and incorrect results.
Best,
Chris
On Feb 22, 2018, at 1:52 AM, Loris Bennett <loris.benn...@fu-berlin.de> wrote:
Hi Chris,
Christopher Samuel <ch...@csamuel.org> writes:
On 22/02/18 18:49, Miguel Gutiérrez Páez wrote:
What's the real meaning of ntasks? Has cpus-per-task and ntasks the
same meaning in sbatch and srun?
--ntasks is for parallel distributed jobs, where you can run lots of
independent processes that collaborate using some form of communication
between the processes (usually MPI for HPC).
So inside your batch script you would use "srun" to start up the tasks.
However, unless you code is written to make use of that interface then
it's not really going to help you, and so for any multithreaded
application you need to use --cpus-per-task instead.
[snip (11 lines)]
But does it make any difference for a multithreaded program if I have
#SBATCH --ntasks=4
#SBATCH --nodes=1-1
rather than
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=4
Up to now I have only thought of --cpus-per-task in connection with
hybrid MPI/OpenMP jobs, which we don't actually have. Thus I tend to
tell users to think always in terms of tasks, regardless of whether
these are MPI processes or just threads.
One downside of my approach is that if the user forgets to specify
--nodes and --ntasks is greater than 1, non-MPI jobs can be assigned to
multiple nodes.
Cheers,
Loris
--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de
