[slurm-users] Inaccurate Preemption Notification?

2023-04-24 Thread Jason Simms
Hello all, A user received an email from Slurm that one of his jobs was preempted. Normally when a job is preempted, the logs will show something like this: [2023-03-30T08:19:16.535] [25538.batch] error: *** JOB 25538 ON node07 CANCELLED AT 2023-03-30T08:19:16 DUE TO PREEMPTION ***

Re: [slurm-users] Terminating Jobs based on GrpTRESMins

2023-04-24 Thread Hoot Thompson
See below…... > On Apr 24, 2023, at 1:55 PM, Ole Holm Nielsen > wrote: > > On 24-04-2023 18:33, Hoot Thompson wrote: >> In my reading of the Slurm documentation, it seems that exceeding the limits >> set in GrpTRESMins should result in terminating a running job. However, in >> testing this,

Re: [slurm-users] Terminating Jobs based on GrpTRESMins

2023-04-24 Thread Ole Holm Nielsen
On 24-04-2023 18:33, Hoot Thompson wrote: In my reading of the Slurm documentation, it seems that exceeding the limits set in GrpTRESMins should result in terminating a running job. However, in testing this, The ‘current value’ of the GrpTRESMins only updates upon job completion and is not

[slurm-users] Terminating Jobs based on GrpTRESMins

2023-04-24 Thread Hoot Thompson
In my reading of the Slurm documentation, it seems that exceeding the limits set in GrpTRESMins should result in terminating a running job. However, in testing this, The ‘current value’ of the GrpTRESMins only updates upon job completion and is not updated as the job progresses. Therefore jobs

Re: [slurm-users] Migration of slurm communication network / Steps / how to

2023-04-24 Thread Ole Holm Nielsen
On 4/24/23 08:56, Purvesh Parmar wrote: Thank you.. will try this and get back. Any other step being missed here for migration? I don't know if any steps are missing, because I never tried moving a cluster like you want to do. /Ole On Mon, 24 Apr 2023 at 12:08, Ole Holm Nielsen

Re: [slurm-users] Migration of slurm communication network / Steps / how to

2023-04-24 Thread Purvesh Parmar
Thank you.. will try this and get back. Any other step being missed here for migration? Thankyou, Purvesh On Mon, 24 Apr 2023 at 12:08, Ole Holm Nielsen wrote: > On 4/24/23 08:09, Purvesh Parmar wrote: > > thank you, however, because this is change in the data center, the names > > of the

Re: [slurm-users] Migration of slurm communication network / Steps / how to

2023-04-24 Thread Ole Holm Nielsen
On 4/24/23 08:09, Purvesh Parmar wrote: thank you, however, because this is change in the data center, the names of the servers contain datacenter names as well in its hostname and in fqdn as well, hence i have to change both, hostnames as well as ip addresses, compulsorily, to given hostnames