Re: [slurm-users] Quick hold on all partitions, all jobs

2017-11-08 Thread John Hearns
case, HOLDing PENDING jobs is redundant.) > > ~jonathon > > > From: slurm-users on behalf of > Lachlan Musicman > Sent: Wednesday, November 8, 2017 5:00:12 PM > To: Slurm User Community List > Subject: [slurm-users] Quick hold on all partitions, all jobs > >

Re: [slurm-users] Quick hold on all partitions, all jobs

2017-11-08 Thread Jonathon A Anderson
run. (In either case, HOLDing PENDING jobs is redundant.) ~jonathon From: slurm-users on behalf of Lachlan Musicman Sent: Wednesday, November 8, 2017 5:00:12 PM To: Slurm User Community List Subject: [slurm-users] Quick hold on all partitions, all jobs T

Re: [slurm-users] Quick hold on all partitions, all jobs

2017-11-08 Thread Christopher Samuel
On 09/11/17 11:00, Lachlan Musicman wrote: > I've just discovered that the partitions have a state, and it can be set > to UP, DOWN, DRAIN or INACTIVE. DRAIN the partitions to stop new jobs running, then you can work on how you suspend running jobs (good luck with that!). -- Christopher Samuel

Re: [slurm-users] Quick hold on all partitions, all jobs

2017-11-08 Thread Stradling, Alden Reid (ars9ac)
We use something like this: scontrol create reservation starttime=2017-11-08T06:00:00 duration=1440 user=root flags=maint,ignore_jobs nodes=ALL Reservation created: root_2 Then confirm: scontrol show reservation ReservationName=root_2 StartTime=2017-11-08T06:00:00 EndTime=2017-11-09T06:00:0

[slurm-users] Quick hold on all partitions, all jobs

2017-11-08 Thread Lachlan Musicman
The IT team sent an email saying "complete network wide network outage tomorrow night from 10pm across the whole institute". Our plan is to put all queued jobs on hold, suspend all running jobs, and turning off the login node. I've just discovered that the partitions have a state, and it can be s