Hi, If you only want 1 job at a time, the best option is to configure the SelectType plugin to select/linear ( https://slurm.schedmd.com/slurm.conf.html#OPT_select/cons_res). That will force slurm to allocate full nodes.
Regards, Carlos On Thu, Apr 30, 2020 at 7:01 PM Allan Streib <astr...@indiana.edu> wrote: > Well looking at the current slurm.conf it appears that the name was > changed, and "Shared" is now called "OverSubscribe" in more modern slurm > versions. So you might look deeper at what config options are in > conflict, since with the EXCLUSIVE mode I get one node per job here. > > Allan > > Allan Streib <astr...@indiana.edu> writes: > > > I'm running an even older slurm than you (it does what I need, I am a > > team of one and I have many things to take care of other than chasing > > the latest version of every piece of software). > > > > Anyway, did you try Shared=EXCLUSIVE in the partition configuration? > > > > From the (v14.11.7) slurm.conf man page: > > > > Shared Controls the ability of the partition to execute more than > > one job at a time on each resource [...] > > > > EXCLUSIVE Allocates entire nodes to jobs even with select/cons_res > > configured. Jobs that run in partitions with "Shared=EXCLUSIVE" will > > have exclusive access to all allocated nodes. (node, socket or core > > depending upon the value of SelectTypeParameters). > > > > Allan > > > > > > Rutger Vos <rutger....@naturalis.nl> writes: > > > >> I did some tests and this doesn't work with MPI, in the following > >> sense: the way we were running parallel jobs was by invoking `sbatch > >> -n4 script.sh` where the script.sh would contain an `mpirun` command > >> without specifying the number of processors. It would then simply claim > >> the ones assigned to it by sbatch. With one CPU as per the slurm.conf, > >> mpirun would only ever get one processor, hence no parallelization. > >> > >> The other test I did was to try OverSubscribe=Exclusive on the > >> partition, but the slurmctld failed to start so there is some > >> combination of config options that it is unhappy about. > >> > >> The workable option I can think of right now is to have everyone run > >> their jobs with --exclusive as an extra argument. > >> > >> On Thu, Apr 30, 2020 at 12:22 PM Rutger Vos <rutger....@naturalis.nl> > >> wrote: > >> > >> This seems like the easiest solution, but how would that interact > >> with MPI jobs? Apologies if these are really basic questions, and > >> thank you very much for thinking along. > >> > >> Rutger > >> > >> On Wed, Apr 29, 2020 at 10:25 PM Doug Meyer <dame...@raytheon.com> > >> wrote: > >> > >> Change node definition in slurm.conf for that one node to 1 > >> CPU. > >> > >> Doug Meyer > >> > >> From: slurm-users <slurm-users-boun...@lists.schedmd.com> On > >> Behalf Of Rutger Vos > >> Sent: Wednesday, April 29, 2020 1:20 PM > >> To: Slurm User Community List <slurm-users@lists.schedmd.com> > >> Subject: [External] Re: [slurm-users] one job at a time - how > >> to set? > >> > >> Hi Michael, > >> > >> thanks very much for your swift reply. So here we would have to > >> convince the users they'd have to specify this when submitting, > >> right? I.e. 'sbatch --exclusive myjob.sh', if I > >> understand correctly. Would there be a way to simply enforce > >> this, i.e. at the slurm.conf level or something? > >> > >> Thanks again! > >> > >> Rutger > >> > >> On Wed, Apr 29, 2020 at 10:06 PM Renfro, Michael < > >> ren...@tntech.edu> wrote: > >> > >> That’s a *really* old version, but > >> > https://slurm.schedmd.com/archive/slurm-15.08.13/sbatch.html > >> indicates there’s an exclusive flag you can set. > >> > >> On Apr 29, 2020, at 1:54 PM, Rutger Vos < > >> rutger....@naturalis.nl> wrote: > >> > >> . > >> > >> Hi, > >> > >> for a smallish machine that has been having degraded > >> performance we want to implement a policy where > >> only one job (submitted with sbatch) is allowed to run > >> and any others submitted after it are supposed to wait > >> in line. > >> > >> I assumed this was straightforward but I can't seem to > >> figure it out. Can I set that up in slurm.conf or in > >> some other way? Thank you very much for your help. BTW > >> we are running slurm 15.08.7 if that is at all > >> relevant. > >> > >> Best wishes, > >> > >> Dr. Rutger A. Vos > >> > >> Researcher / Bioinformatician > >> > >> [logo-new] > >> > >> [+31717519600 - +31627085806 ] > >> > >> [rutger....@naturalis.nl - www.naturalis.nl] > >> > >> [Darwinweg 2, 2333 CR Leiden] > >> > >> [Postbus 9517, 2300 RA Leiden] > >> > >> [schildpad] > >> > >> -- > >> > >> Met vriendelijke groet, > >> > >> Dr. Rutger A. Vos > >> > >> Researcher / Bioinformatician > >> > >> [logo-new] > >> > >> [+31717519600 - +31627085806 ] > >> > >> [rutger....@naturalis.nl - www.naturalis.nl] > >> > >> [Darwinweg 2, 2333 CR Leiden] > >> > >> [Postbus 9517, 2300 RA Leiden] > >> > >> [schildpad] > >> > >> -- > >> > >> Met vriendelijke groet, > >> > >> Dr. Rutger A. Vos > >> Researcher / Bioinformatician > >> [logo-new] > >> > >> [+31717519600 - +31627085806 ] > >> [rutger....@naturalis.nl - www.naturalis.nl] > >> [Darwinweg 2, 2333 CR Leiden] > >> [Postbus 9517, 2300 RA Leiden] > >> > >> [schildpad] > >> > >> -- > >> > >> Met vriendelijke groet, > >> > >> Dr. Rutger A. Vos > >> Researcher / Bioinformatician > >> [logo-new] > >> > >> [+31717519600 - +31627085806 ] > >> [rutger....@naturalis.nl - www.naturalis.nl] > >> [Darwinweg 2, 2333 CR Leiden] > >> [Postbus 9517, 2300 RA Leiden] > >> > >> [schildpad] > >> > > -- -- Carles Fenoy