Re: [slurm-users] Slurm node history / log ?

2023-07-05 Thread Ford, Steven
Hi Bill, I think the command you're looking for is `sacctmgr show event`. Best, Steve From: slurm-users On Behalf Of Bill Benedetto Sent: Wednesday, July 5, 2023 13:21 To: slurm-users@lists.schedmd.com Subject: [slurm-users] Slurm node history / log ? Good day. Is there some command that I

Re: [slurm-users] Slurm node history / log ?

2023-07-05 Thread Roberto Monti
Hi Bill, Your best bet is probably /var/log/slurmctld on the server that is acting as active controller. Best, -- Roberto P. Monti DevOps Engineer I roberto.mo...@jax.org The Jackson Laboratory United States | China | Japan www.jax.org From: slurm-users On Behalf Of Bill Benedetto Sent:

[slurm-users] Slurm node history / log ?

2023-07-05 Thread Bill Benedetto
Good day. Is there some command that I can use in Slurm to see a node's history? Not the job history, but the state history. Something like: Jul 5 13:11:01 node01 taken offline by slurmctld because node01 not responding And/Or: Jul 5 13:11:01 node01 taken offline by USER1 state=DRAIN

Re: [slurm-users] Slurm commands for launching tasks: salloc and sbatch

2023-07-05 Thread Mike Mikailov
Hi Paul,Thank you for the explanations. Actually this was not the main point of the question asked. I think we can close the discussions. The main point was: why a job is running more efficiently using sbatch than salloc.Thank you all for the contributions. - MikeSent from my iPhoneOn Jul 5, 2023,

Re: [slurm-users] AllowGroups for Partition not working?

2023-07-05 Thread Xand Meaden
You should check whether the relevant group's members can be seen using command `getent group `. If not, you probably need to add/change the "winbind expand groups" option in smb.conf. Xand On 05/07/2023 17:17, Matthias Leopold wrote: Thanks, but unfortunately that didn't help. Regards,

Re: [slurm-users] Slurm commands for launching tasks: salloc and sbatch

2023-07-05 Thread Paul H. Hargrove
Mike, I think your definitions are probably in the minority on this list. To be clear, I am *not* saying you (or SGE) are wrong, just that the folk here use different terms for what you are asking for. I think of it like dialects of English where the same food might be a "cookie" or a "biscuit"

Re: [slurm-users] AllowGroups for Partition not working?

2023-07-05 Thread Matthias Leopold
Thanks, but unfortunately that didn't help. Regards, Matthias Am 05.07.23 um 17:59 schrieb Marko Markoc: Hi Matthias, Before you start digging deeper into this, I would recommend restarting the `slurmctld` service. I've had similar issues where `scontrol reconfigure` wasn't enough for

Re: [slurm-users] AllowGroups for Partition not working?

2023-07-05 Thread Marko Markoc
Hi Matthias, Before you start digging deeper into this, I would recommend restarting the `slurmctld` service. I've had similar issues where `scontrol reconfigure` wasn't enough for certain configuration changes. Regards, Marko On Tue, Jul 4, 2023 at 3:57 AM Matthias Leopold <

Re: [slurm-users] Slurm commands for launching tasks: salloc and sbatch

2023-07-05 Thread Mike Mikailov
Thank you Loris, for the further feedback. “Reasonable” for SGE is within a few minutes, would be nice if it could be adjusted. Still interactive means the user has almost immediate access to the system, not queued. Sent from my iPhone > On Jul 5, 2023, at 9:43 AM, Loris Bennett wrote: >

Re: [slurm-users] Slurm commands for launching tasks: salloc and sbatch

2023-07-05 Thread Loris Bennett
Mike Mikailov writes: > Thank you Loris, for the further clarifications. The only question is > who will wait forever in interactive mode? And how practical is it? > > Interactive mode is what its name implies - interactive, not queueing. To me, "interactive" is the alternative to "batch" -

Re: [slurm-users] Problem with Cuda program in multi-cluster

2023-07-05 Thread Feng Zhang
Mohamad, It seems you need to upgrade the GCC on the GPU nodes of cluster A and C. The error message says that the srun needs newer GCC libs. Or you can downgrade your SLURM(like recompile it using GCC 2.27 or older) on cluster A/C. Best, Feng On Tue, Jul 4, 2023 at 2:46 PM mohammed shambakey

Re: [slurm-users] Slurm commands for launching tasks: salloc and sbatch

2023-07-05 Thread Mike Mikailov
Thank you Loris, for the further clarifications. The only question is who will wait forever in interactive mode? And how practical is it? Interactive mode is what its name implies - interactive, not queueing. It would make more sense if the default setting for deadline would be set to a